Multiresolution processing of satellite images and geographic information systems techniques for land-use classification

MISSING IMAGE

Material Information

Title:
Multiresolution processing of satellite images and geographic information systems techniques for land-use classification
Physical Description:
xiii, 224 leaves : ill. ; 29 cm.
Language:
English
Creator:
Tan, Yurong
Publication Date:

Subjects

Subjects / Keywords:
Agricultural Engineering thesis Ph. D
Dissertations, Academic -- Agricultural Engineering -- UF
Genre:
bibliography   ( marcgt )
non-fiction   ( marcgt )

Notes

Thesis:
Thesis (Ph. D.)--University of Florida, 1994.
Bibliography:
Includes bibliographical references (leaves 213-219)
Statement of Responsibility:
by Yurong Tan.
General Note:
Typescript.
General Note:
Vita.

Record Information

Source Institution:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 002043693
oclc - 33327857
notis - AKN1588
System ID:
AA00004730:00001

Full Text









MULTIRESOLUTION PROCESSING OF SATELLITE IMAGES AND
GEOGRAPHIC INFORMATION SYSTEMS TECHNIQUES FOR
LAND-USE CLASSIFICATION


















By

YURONG TAN


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


1994


































Copyright 1994

by

Yurong Tan












ACKNOWLEDGEMENTS


This research was performed using the facilities of the

Remote Sensing Applications Laboratory (RSAL) of the

Department of Agricultural Engineering at the University of

Florida. The author is grateful for the assistance provided

by the RSAL director, Dr. Sun F. Shih; RSAL manager, Orlando

Lanni; and RSAL assistants, Jonathan D. Jordan, Chih-Hung Tan,

and Bruce E. Myhre. The author is also grateful for the

comments and review of this manuscript by Dr. Donald L. Myhre

of the Soil and Water Science Department at the University of

Florida.

The author extends his appreciation to each of the

members of his advisory committee, Dr. Brian J. Boman, Dr.

Edward P. Lincoln, Dr. Allen R. Overman, and Dr. Byron E. Ruth

for their comments and advice rendered during this research,

particularly to the committee chairman, Dr. Sun F. Shih of the

Department of Agricultural Engineering at the University of

Florida, for the guidance and support provided throughout the

course of his graduate study at the University of Florida.

The greatest gratitude of the author goes to his wife,

Siru, who always shared the joy as well as frustrations and

provided unconditional support in every aspect during this

seemingly endless process. The author is in debt to his son,

Guolong, for the patience he undertook while this research was

being actively pursued.


iii








Lastly, the author acknowledges that this research would

not have been completed without the continuous support and

encouragement from his loving parents.












TABLE OF CONTENTS


ACKNOWLEDGEMENTS ...................................... iii

LIST OF TABLES .......................................... vii

LIST OF FIGURES ......................................... ix

ABSTRACT ................................................. xii

CHAPTERS

1 INTRODUCTION .................................. 1
Overview ................. ................. 1
Statement of Research Problem ................. 3
Concept of Multiresolution Processing ....... 8

2 OBJECTIVES OF RESEARCH .......................... 13

3 REVIEW OF LITERATURE .......................... 14
Conventional Image Enhancement .............. 14
Multiresolution Enhancement ................ 20
Color Composite Generation .............. 20
Radiometric Enhancement ................. 27
Other Enhancement Methods ............... 36
Summary: Assessment of Problems ............. 40

4 PRINCIPLE OF MERGING IMAGES ..................... 46
Principle of Merging Images ................. 46
Assumptions ............................. 47
Arithmetic of Random Variables ......... 48
Confining Method ........................... 53
Preserving Method .......................... 63
Differencing Method ........................ 69
Summary: Principle of Merging Images ........ 73

5 DEMONSTRATION OF MERGING METHODS ................ 76
Satellite Image Data ......................... 76
Variance of Merged LAC Images ............. 81
Comparison of Merged LAC Images ............ 91
Ratioing of Satellite Images ................ 106
Multiresolution Enhancement ................. 113
Summary: Appraisal of Merging Methods ....... 116

6 MATERIALS AND METHODOLOGY FOR
MULTIRESOLUTION LAND-USE CLASSIFICATION ........ 119
Data Source and Equipment ................... 119








SPOT Image Data and Study Area .......... 119
ACIR Photography ....................... 122
Image Processing Systems ................ 122
Photogrammetric Stereo Plotter .......... 123
Procedures for Merging SPOT Dataset ......... 124
Pre-merging Processing .................. 124
Generating Merged Dataset ............... 126
Evaluation of Merged Data ............... 129
Image Response and Citrus Canopy Cover ...... 132
Photogrammetric Measurement ............. 132
Canopy Cover Estimation ................. 134
Land-use Classification ..................... 136
Precis and Concept ....................... 137
Extracting Signature Patterns ........... 139
GIS-base Discrete Classification ........ 141

7 DISCUSSIONS AND ANALYSES OF
MULTIRESOLUTION LAND-USE CLASSIFICATION ......... 148
Evaluation of Merged Image ................. 148
Radiometric Quality ..................... 148
Spatial Improvement and Spectral
Integrity .......................... 151
Image Response and Citrus Canopy Cover ...... 157
Estimation of Citrus Canopy Cover ....... 157
Relation of Image response to
Canopy Cover ....................... 159
Differentiation of Canopy Cover ......... 166
Effect of Multiresolution Merging ...... 177
Land-use Classification ..................... 183
Potential for Signature Extraction ...... 184
GIS Discrete Classification ............. 189

8 CONCLUSIONS AND RECOMMENDATIONS ................. 195
Research Conclusions ....................... 195
Recommendations ............................ 199

APPENDICES

A RGB COLOR DISPLAY ............................... 202

B IHS TRANSFORM FOR IMAGE DISPLAY ................ 203

C PROGRAM CODES TO UNPACK AVHRR LAC DATA .......... 206

D CLASSIFICATION DECISION RULES .................. 209

REFERENCES ......... .................................... 213

GLOSSARY ................................................ 220

BIOGRAPHICAL SKETCH .................................. 224












LIST OF TABLES


Table

1-1 Available sources of Landsat and SPOT resource
satellite data and system characteristics ....... 5

4-1 Summary of the characteristics of different
merging approaches ............................ 75

5-1 Wavelength characteristics of NOAA-11
AVHRR LAC images ................................ 77

5-2 standard deviation (a), normalized variance
(g2), mean (A), maximum and minimum values
of NOAA-11 AVHRR LAC images .................... 80

5-3 Offset constant (C) used in the differencing
method for merging LAC images ................... 82

6-1 Standard deviation (a), mean (.), and maximum
and minimum values, and correlation coefficients
(r) of SPOT multiresolution dataset ............. 127

6-2 Multiresolution datasets and corresponding
merging equations .............................. 130

6-3 Parameters used in ERDAS STATCL and ELAS
TMTR modules for signature extraction ........... 142

7-1 Standard deviation (a) and mean brightness
values (I) for multiresolution merged SPOT
images ... .... ................................... 149

7-2 Summary for correlations between a merged image
and its original multispectral counterpart ...... 153

7-3 Between-waveband correlations (r) within
multiresolution merged datasets ................. 154

7-4 Summary for corelations between citrus
canopy size and image response for
multiresolution merged images ................. 178

7-5 Variation of image data correlation (r)
between panchromatic and original
multispectral wavebands among selected groves ... 179


vii








7-6 Standard deviations (a) of merged image
data for selected citrus groves ................. 180

7-7 Summary of spectral signatures unveiled
by ERDAS STATCL module ........................ 185

7-8 Summary of spectral signatures unveiled
by ELAS TMTR module ........................... 188

7-9 Canopy cover for spectral classes by GIS-
based discrete classification technique ......... 190


viii












LIST OF FIGURES


Figure Bue

1-1 Schematics of merging multiresolution
satellite images ............................... 9

3-1 Schematics of principal component analysis
for multispectral datasets .................... 18

4-1 Relation of radiometric variance to merging
coefficient (B) and correlation coefficient
(r) for the confining method .................... 57

4-2 Effect of variance difference on the
radiometric quality of merged images
for the confining method ........................ 61

4-3 Relation of radiometric variance to merging
coefficient (B) and correlation coefficient
(r) for the preserving method ................... 66

4-4 Effect of variance difference on the
radiometric quality of merged images
for the preserving method ....................... 68

5-1 Location of clipped NOAA-11 AVHRR LAC images .... 79

5-2 Comparison between actual and estimated
radiometric variance for merged LAC
images (case I) ............................... 83

5-3 Comparison between actual and estimated
radiometric variance for merged LAC
images (case II) ................................ 84

5-4 Comparison between actual and estimated mean
digital count for merged LAC images (case I) .... 85

5-5 Comparison between actual and estimated mean
digital count for merged LAC images (case II) ... 86

5-6 Original clipped NOAA-11 LAC images of
red and NIR wavebands .......................... 93

5-7 Merged LAC images by the preserving
method (case I) ................................. 94








5-8 Merged LAC images by the preserving
method (case II) ................................ 96

5-9 Merged LAC images by the confining
method (case I) ................ ................. 97

5-10 Merged LAC images by the confining
method (case II) ............................... 98

5-11 Merged LAC images by the differencing
method (case I) ................................ 100

5-12 Merged LAC images by the differencing
method (case II) ............................... 101

5-13 Summary (mosaic) of merged LAC images for
three methods (case I) .......................... 104

5-14 Summary (mosaic) of merged LAC images for
three methods (case II) ......................... 105

6-1 Location of clipped SPOT multiresolution
dataset and study area ......................... 121

7-1 Comparison of SPOT 20-m NDVI and 10-m
NDVIp images .................................... 156

7-2 Effect of citrus canopy cover on SPOT green
waveband response .............................. 160

7-3 Effect of citrus canopy cover on SPOT red
waveband response ............................... 161

7-4 Effect of citrus canopy cover on SPOT
panchromatic waveband response .................. 163

7-5 Effect of citrus canopy cover on SPOT NIR
waveband response .............................. 164

7-6 Coincident plot of SPOT green waveband
response for select citrus groves ............... 168

7-7 Coincident plot of SPOT red waveband
response for select citrus groves .............. 169

7-8 Coincident plot of SPOT NIR waveband
response for select citrus groves ............... 170

7-9 Effect of tree crown variations on SPOT
green waveband response variability for
partial canopy groves .......................... 172








7-10 Effect of tree crown variations on SPOT
red waveband response variability for
partial canopy groves ......................... 173

7-11 Effect of tree crown variations on SPOT
NIR waveband response variability for
partial canopy groves ....................... 174

7-12 Relation of citrus tree variations to
canopy cover difference ........................ 176












Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

MULTIRESOLUTION PROCESSING OF SATELLITE IMAGES AND
GEOGRAPHIC INFORMATION SYSTEMS TECHNIQUES FOR
LAND-USE CLASSIFICATION

By

Yurong Tan

December, 1994


Chairman: Dr. Sun Fu Shih
Major Department: Agricultural Engineering

Combining multiresolution images to improve land-use

information assessment is an important subject in remote

sensing applications. The problem in finding effective

methods for multispatial processing must be resolved through

the development of new procedures for merging satellite

images. The effect of combining image data on the quality of

merged datasets must also be assessed toward land-use

classification applications and multispectral analyses.

In digitally merging satellite images, the statistical

variation analyses for combining random variables can be used

to understand the various forms of image data merging and

assess the radiometric quality of pre-merged images. The

selection of an effective merging approach must be made with

consideration of both the correlation and radiometric variance

difference between the combining images and the merging

coefficients. Merging images is a radiometric transformation


xii








among the various land-use types in a scene. This principle

can be used to collaborate Landsat MSS, Landsat TM, SPOT, and

other satellite data for broader applications.

To generate enhanced datasets, the preserving approach

should be used for non-negatively correlated images, and the

differencing approach for those with negative correlations.

The commonly used, but ineffective confining method should be

avoided. The efficacy of waveband ratioing is limited to the

land-use elements with weak/negative correlations and larger

values in the numerator image.

The preserving method with a B=0.5 coefficient was

effective in generating both spatially and radiometrically

enhanced SPOT multiresolution merged datasets which

consistently rendered significantly more spectral signatures

from a satellite scene. This enhanced differentiation

provides a greater amount of information for applications

including land-use classification and image interpretation.

The photogrammetric estimation of citrus canopy cover is

feasible and accurate. Except for the NIR waveband, citrus

canopy cover is inversely related to SPOT image spectra of

partial canopy groves, suggesting a strong influence of soil

substrate on satellite image response. The canopy-size

classification of citrus groves was improved through the

combined use of merged SPOT dataset and GIS-based

classification techniques. Citrus groves with a higher

percentage of canopy cover had more uniform trees and less

variable spectral responses.


xiii












CHAPTER 1
INTRODUCTION


Overview


Information about land use plays an increasingly

important role in the management and preservation of natural

resources. For instance, land-use data are used in the

operations of water resources management which range from

water-use permitting to the development and implementation of

regional planning and management strategies. In environmental

and water quality monitoring, land-use activities are often

indicative of the source and type of pollutants (Novotny and

Chesters, 1981; Fukushima and Muraoka, 1988), particularly

from agricultural and urban lands (USEPA, 1984; Pionke and

Urban, 1985). In many cases, it is the change of land use

that creates immense environmental concerns. Agricultural

land-use data are needed to forecast and monitor production as

well as to assess damage caused by diseases and natural

catastrophic events. Also, land-use data are used in many

other ways including forest management (Coleman et al., 1990),

urban development and planning (Colwell and Poulton, 1985),

hydrological investigations, and applications of geographic

information systems (Ehlers, 1989; Piwowar et al., 1990; Tan

and Shih, 1991a). Therefore, the availability of quality and

timely land-use information becomes an indispensable factor

1








2

which prescribes our efforts in better managing natural

resources.

Traditionally, land-use data are collected through aerial

photography, ground surveys, and existing maps. While these

methods are reliable and accurate, they are expensive and

time-consuming. In addition, the process of traditional

methods is tedious, and therefore often provides land-use data

that are years out of date, while data availability becomes a

limiting factor in some cases. When a large coverage area is

needed, the difficulties involved increase in magnitude as

well as in complexity. Fortunately, the synoptic coverage and

periodic availability of satellite remote sensing data provide

an excellent opportunity for the acquisition of timely land-

use data and the monitoring of extensive land-use activities.

This significantly amplifies our ability to understand the

effects of land use types and to manage the impacts and

consequences resulting from the change of land use activities.

With increasing environmental awareness, more careful planning

and monitoring of land-use activities becomes an important

consideration in all levels of resources management.

To derive land-use information from satellite data, a

land-use classification procedure is used within an automated

computer image processing system. Such procedures generate

statistically similar spectral classes which are then related

to different land-use types (Lillesand and Kiefer, 1979;

Thomas et al., 1987) through a ground-truthing process. To










improve the acquisition of land-use information from space,

continuous research efforts are underway in the development of

both new sensing systems (Engel, 1986; Spotlight, 1991; EOSAT,

1992a, 1992b) and image processing techniques.


Statement of Research Problem


Obtaining land-use data or land-use information by

satellite remote sensing requires a significant improvement

both in accuracy and in specificity in order to be used

operationally in many applications (Lo et al., 1986; DeGloria

et al., 1986). For instance, day-to-day operations in water

resources management seldom use satellite-based land-use data,

mainly because of the lack of desired specificity or details.

One facet to the solution of this problem is to improve the

quality of raw data through advanced sensing technology and

sensor system design. This has been initiated by the

development of new sensing systems which will be onboard

Landsat-7 (EOSAT, 1992a, 1992b) and the French Systeme Probatoire

de 1'Observation de la Terre (SPOT) resources satellite four

referred to as SPOT-4 (Spotlight, 1991). Equally important is

the development of data processing techniques to analyze and

classify the remotely sensed data so that improved land-use

information becomes feasible in practical applications.

Combining multispectral satellite data that have

different spatial resolutions to extract more subtle land-use

information has become an important component in image








4

processing techniques. In the process, the spectral and

spatial advantages rendered by different sensing systems

(Table 1-1) are combined complementarily into a merged

dataset. This provides an unparalleled opportunity that

expands our ability beyond using any of the original

individual datasets to acquire land-use information. Because

of the challenge of future sensor systems which will provide

multiresolution sensing as well as onboard registration

capabilities (Spotlight, 1991; EOSAT, 1992a, 1992b) and the

tremendous amount of image data already captured by satellite

sensors operating over a wide range of spatial resolutions and

spectral wavebands (Shih, 1984; Moore, 1989; Ehlers, 1989),

merging multiresolution satellite images creates an immense

opportunity to make contributions to the improvement of

current land-use data acquisition from space. As a result,

multiresolution processing is anticipated to be a very

powerful image processing technique in future remote sensing

applications.

To date, much research work remains to be done in order

to effectively use multiresolution satellite imagery for

resources management. For instance, finding effective methods

to digitally merge multiresolution datasets continues to be

the central problem in multiresolution processing. A good

merger will be able to take full advantage of the spectral and

spatial benefits of multiresolution images so that resultant

merged datasets will have incomparable radiometric quality















0
C

0

4-I
,,a
CO 4
^i


t)
>1
CIn


41
0


4)


0




E43






U)


In







(4
0)










I


0






*4
U)
S(



to
(0







*oI










44
0









I








rO
0,








I$
*1* 1


ar o o




















k 54 4
O I 00 $4 (
$4 $4 4
HI WW rW Id
1 o 0-I- g









t- CO) 0000 I




w C 0000 1
SC1 4 orW)o i) %D I


moo











r4(4CI I


4.4
0 W
0
O


o t-r-o ld to I
S 4I4 4
V I













a


k .--I r

0 r. r- VI

4- r4 to

$0
k 4) 4 ) 1

4 9 ZZZ E-4 4)
0
I



0
LOI

0 000 HCN I
I H I
0

0 00000 1
H I
(d0
ku
CC 4 IC
O ##94


000

























0 4 10
S1 W
(Q U O


00
I I



0 0
00
















Eu -v.4
O 3
(v .-I 4
10 6
0)
w w'--


0000 D0 0DD0D0
NI










0 0


$.4 -4


0 4) $4 4) 4) ) )


44 #0 4O t-l 4


$4 4) EQ) ) k M
A 9 4 9 k ) 4) 4) 1
Z HE(W-4 H .
c-4 0 0 0
0 1II










0 r- NOD 0000 1
*... L n~orco I
1-4* *0 O H I

4909 0000
-W tnqo %am q I
OrION 0000 I


ON
0 *
00


000


I I I I I I


o 00
* *
o o


(N CO
888Ol


I I



-ON
1-4


M0000000
i-inmmmn




M -4



o to


SMd
r 4) )

0 4) 4 4) 4Q)

f *il rq) *r4


44 MV V


0PMOIPZHE-4H


0 M
O
0

E-C


0000
-B N N VN


















o )4




O! C
S0 9
C00 I
9 Wuo 0


O % coo

000


0 -1 0

000


Hi N m -q i 0 r














0

WH
toH 0
mO (
me B


M' co(A Ln

0 00 r-


o C--i co
*OO O


0 0000
















SV
(0










4 00)
U 4 ) Qa J
( 0)C
0< UK SSHi


**rl

A)
0
00

43
4.)

44




o i
i> oco 3JO> *A
SC 0C M\COC
\\\' 0) Q0 0 0 0 W-C

O\\\ 0I NM LV
c-l N Ln OD 04 -0 H D0



0% ON 0%0 ON (A Or4t
0t 0
1I IV 1 I 4J I I I U
NtncO W HwOO u



N eqO 0 0 0 E NN-4
*\S.\Ns5%.\ <0 V s.NN. 0





0090000r,222
4M44444HNOW









$H







0
o




fz


8
0

o E
0 0
4 OW


o 4


0)



4 4
a


o









t o
rI --
t >

0

UC 4







.r m ) .c
-04


> C0 -4
0 C b--1



to U pI 4 4
4 U U CH
>R a) )U^-



0 L-i 040 c4



43 o al U -p 0
Li-lo N 0

0I 0 E- E-4 -H

W .Q C 0 *o 40


4-I
ONi
0
0
O
4)
tO








8

and enhanced spatial information. These quality factors of

image data are vital not only to the utility of the merged

datasets for potential applications, but also to any remote

sensing efforts that attempt to improve our capability in

monitoring land-use resources. Therefore, this research was

focused on the techniques for combining multiresolution

satellite images as well as on the utility of multiresolution

processing for land-use classification and image

interpretation.


Concept of Multiresolution Processing


Multiresolution processing is an image processing

technique used to combine or merge multispectral images that

have different spatial resolutions. One of the images to be

combined will have a high spatial resolution or a smaller size

of picture element (pixel), and a panchromatic waveband, while

the others will be multispectral (or multi-waveband), but with

a relatively lower spatial resolution, or a larger pixel.

These images of different spatial resolutions are digitally

merged to reconstruct a new set of images that can inherit the

spectral and spatial characteristics of both the multispectral

and panchromatic images. The process is schematically

illustrated in Figure 1-1.

The purpose of multiresolution processing is to generate

a new set of images with enhanced spectral and spatial

qualities by taking the spectral and spatial advantages of the




















r-4


-0
M
U



-0
a









a,


E
0


CM

0.


I.


z
C-


z
0.


0


-,4
't




4.
ta






0




(0
4
DC


z
a->


I








10

images to be combined. The spatial and spectral virtues of

the original multiresolution image data are utilized

complementarily. As a result, the merged dataset becomes

spectrally as well as spatially more powerful for remote

sensing applications.

One example of multiresolution processing is to merge the

SPOT high resolution visible (HRV) panchromatic and

multispectral images that have respective 10-m and 20-m

spatial resolutions (Cliche et al., 1985; Carper et al.,

1990). The panchromatic image with a 10-m resolution can

reveal subtle spatial details of scene objects, but its usage

for multispectral analyses (land-use classification) is

hampered by its very broad spectral waveband. The 20-m

multispectral data with three spectral wavebands are more

useful for land-use classification, however a high spatial

resolution multispectral dataset is more desirable for

extracting subtle information from the scene. When such

multiresolution images are merged, the spectral and spatial

advantages are combined into a new set of images which are

multispectral and with a 10-m spatial resolution. As a

result, the new merged dataset will have a greater potential

for remote sensing applications.

There are two major steps involved in the process of

multiresolution processing. The first deals with the co-

registration of the multiresolution images. This can be done

with two different approaches. The first one is to simply







11

project all images to a mutual geographic reference system

which can be either the latitude-longitude system, or the

universal transverse mercator (UTM) system, or the state plane

coordinate (SPC) system. The second approach is to treat one

of the images as a master and the rest as slaves. After

selecting a number of tie points that are mutual to all images

including the master one, the slave images are rectified to

the master image. In the second approach, no actual

geographical coordinate system (e.g. UTM) is utilized and the

slave images are referenced relative to the master image.

Usually, this relative approach produces a smaller error of

co-registration because transitional reference (e.g. maps) and

digitizing operations for map data entry are not involved. To

register multiresolution images, it is also necessary to

invoke an image resampling procedure before or during the

registration process. Virtually all image processing software

packages provide the facilities for image resampling and

registration operations.

The second step involves the use of mathematical

manipulations to digitally combine, pixel by pixel, the

numerical image data. This is a very critical step because

the spectral, radiometric, and spatial qualities of the merged

dataset depend on the selection of a good combining algorithm.

At the end, a new set of multispectral images are generated

which are radiometrically, spatially, and spectrally enhanced.

While the combining algorithms reported in the literature vary








12

considerably, their introduction is the results of

speculations and arbitrary elaborations because the basic

principle of digitally merging satellite images is not well

understood. Therefore, it has become essential to explore and

to understand the principle of image data manipulations so

that the techniques of multiresolution processing can be

developed to effectively enhance satellite remote sensing

applications including land-use classification.











CHAPTER 2
OBJECTIVES OF RESEARCH


The main objective of this research was to study the

principle of digitally merging satellite images and to develop

techniques for combining multiresolution satellite datasets,

as well as to evaluate the utility of multiresolution

processing for satellite-based land-use classifications. The

specific objectives included:

1. To formulate the principle of digitally combining

multispectral satellite images including those with different

spatial resolutions.

2. To develop techniques and methods for digitally

merging multiresolution satellite images.

3. To study the effects of multiresolution processing

on the spectral, spatial, and radiometric qualities of merged

multispectral datasets.

4. To study the effects of canopy sizes of citrus trees

on the spectral responses of SPOT satellite data as well as to

investigate the feasibility of a canopy-size differentiation

of citrus crops on satellite images.

5. To investigate the utility and benefits of combining

multiresolution satellite images for land-use classifications,

particularly for citrus crops.












CHAPTER 3
REVIEW OF LITERATURE


Because of the multispectral capabilities of contemporary

satellite sensing systems, a geographic area can be imaged

simultaneously with a number of spectral wavebands, resulting

in a multi-image scene or dataset. In the discussions

throughout this dissertation, a scene includes all the images

acquired for one geographic area at one time, while an image

is meant to represent the numerical data of only one spectral

waveband.


Conventional Image Enhancement


Over the years that remote sensing data have become

widely used, many techniques for image enhancement have been

well developed and standardized in image processing software

systems. Therefore, an in-depth discussion for each of these

techniques seems inappropriate. However, a brief review of

those techniques which were involved or used in many previous

research efforts to combine multiresolution satellite images

would be useful to the understanding of continued discussions.

Those techniques included the contrast stretching, spatial

filtering, and principal component analysis.

A contrast-stretching procedure is an image processing

procedure used to arbitrarily rescale a set of image gray








15

shades to a larger range or to a full range (0-255) for

increasing image contrast. The gray shades of an original

satellite image of Landsat, SPOT, and other satellites usually

spread over a portion of the available 0-255 dynamic range (or

sometimes called data depth). As a result, such images with

cramped gray shades do not have conspicuous tonal gradations.

After a contrast-stretching process, the image values (often

called image digital counts) of relatively dark pixels are

scaled back further, while those of bright pixels are scaled

up. As a result, original dark pixels will become darker

while the bright ones become brighter in a contrast-stretched

image (Gonzalez and Wintz, 1987). Very often, this simple

procedure can produce satisfactory results for image

interpretation. Note that the increase in image tonal

contrast by a contrast-stretching procedure gives a false

sense that the procedure can improve the image radiometric

quality.

The contrast-stretching procedure can be applied to a

portion of the existing image gray shades (Thomas et al.,

1987) or to a subimage area (Gonzalez and Wintz, 1987) for a

selective enhancement. Also, there are linear and nonlinear

contrast stretching methods (Lillesand and Kiefer, 1979;

Thomas et al., 1987) and the procedure is performed for each

image or waveband independently.

A spatial filtering procedure includes both the high-pass

and low-pass filters which are often used to remove or to








16

emphasize certain visual effects of a digital image. For

instance, a low-pass filter is used for image smoothing and

noise elimination while a high-pass filter is for edge

enhancement (Lillesand and Kiefer, 1979). The simplest form

of a low-pass filter is to replace the value of a pixel by the

average computed from its neighborhood (e.g. 3 x 3 pixel

array). By replacing a pixel's value with its neighborhood

average, the large values (such as noises) will be compressed

while the small values are inflated or exaggerated (Lillesand

and Kiefer, 1979). As a result, a low-pass filtered image

will appear smoother and have less contrast. In the case of

a high-pass filter, the value of a pixel will be added to or

subtracted from by its deviation from the average of its

neighborhood (e.g. 3 x 3 array), depending on its relative

magnitude with respect to the defined neighborhood average.

Therefore, boundary pixels which usually have the largest

deviations will become either much darker or brighter. Often,

the deviations are doubled or even tripled in order to make

edges or linear features more conspicuous (Lillesand and

Kiefer, 1979). The operation of a spatial filtering procedure

(high-pass or a low-pass) is performed independently for each

image or waveband. An important point in spatial filtering is

that the resultant image data are radiometrically altered by

such filtering procedures.

As compared to the methods of both contrast stretching

and spatial filtering, principal component analysis (PCA) is








17

a procedure which involves a multi-dimensional transformation

for a set of multispectral images. In the process, the multi-

waveband data are transformed from the original coordinate

system formed by the spectral wavebands into one defined by

new synthesized wavebands. There are several usages for a PCA

procedure. First, it can be used to reduce the dimensionality

of multi-waveband datasets (Thomas et al., 1987). For

instance, when a PCA transform is applied, the image data of

a two-waveband dataset can be effectively represented by the

first principal component (PC1) as shown in Figure 2-1, thus

reducing the dataset to essentially one dimension (or one

synthesized waveband). The second usage is to increase the

image contrast as well as the separability for land-use

elements (Lillesand and Kiefer, 1979). For instance, the

image data variance encompassed by the PC1 component (Figure

2-1) is greater than either of those for the two original

wavebands. Therefore, the image of the PC1 component will

have more contrast as well as greater separation among the

different land-use elements in the image. The third usage of

a PCA procedure is for the decorrelation of multispectral

images (Gillespie et al., 1986). In such a case, a PCA

transform is followed by a contrast-stretching procedure

applied to the PC components, particularly the PC2 component

as shown in Figure 2-1. Then, the first (PC1) and second

(PC2) components are together retransformed back to their

original multispectral space. In a decorrelated dataset, the












a









Sto
r(0
0


.0

AV U
p4 0


















-'-4



.r4
@ e 0*



*.w 0



0 \* o ,








.o; o
3 0


4.-)
-" U







H *
3 ^ *A "*-**0
h V ^ '.4








19

identities for the various types of spectral elements may be

significantly different from those of the original images

(Gillespie et al., 1986).

If the PCA procedure is applied to a multispectral image

dataset with n wavebands, the image data transformation will

take place within a n-dimensional space. The results are that

the amount of radiometric information represented by the

first, the second, ... and the nth component will be in a

decreasing order. Also, the transformed components can each

be contrast-stretched (Lillesand and Kiefer, 1979) to further

enhance the tonal gradations of transformed images. The PCA

transform is usually carried out before initiating land-use

classification procedures to reduce data dimensionality as

well as to enhance the radiometric separability of spectral

classes. Thomas et al. (1987) presents in-depth discussions

about PCA transforms which are exemplified by using a Landsat

multispectral scanner (MSS) dataset.


Finally, it is worthwhile to point out that the

procedures of contrast stretching, spatial filtering, and

principal component analysis will not enhance or improve the

spatial resolution of the original images. In addition, a

contrast-stretching procedure will not increase the actual

number of gray shades in an image, even though the radiometric

variance of stretched image data is increased.









Multiresolution Enhancement


Combining the spatial and multispectral advantages of

multiresolution satellite images for various resource

management applications has inspired great interests in the

remote sensing community. To merge satellite images with

different spatial resolutions, both image co-registration and

arithmetical data manipulations are required. If the images

are already co-registered, the primary methods to manipulate

the image data can be summarized into two broad approaches

which include the generation of color composites and the

enhancement of radiometric quality of merged images. Note

that to generate a color composite usually requires three

images for the blue, green, and red primary colors of a

display device.


Color Composite Generation


The most commonly used methods for generating color

renditions from image datasets are the well-known RGB (red,

green, and blue) color display system (Appendix A) and the

intensity-hue-saturation (IHS) color transform (Appendix B).

For multiresolution datasets, the RGB system is simple and

easy to use, but the resultant color composites often have a

blocky appearance, particularly when the spatial resolution

difference is large. Recently, the IHS transform has gained

popularity mainly because of its effectiveness to produce more

balanced color products for a wide range of datasets. To








21

generate a color composite from a multiresolution dataset, the

IHS method first takes a forward transformation from the low

spatial resolution images of three wavebands into the

intensity (I), hue (H), and saturation (S) components

(Appendix B). Then, a reverse transformation is carried out

to convert the I, H, and S components to the RGB values in

order to generate color composites through a RGB color display

device. The high spatial resolution image is merged in the

process by replacing the I component during the reverse

transformation (Haydn et al., 1982; Carper et al., 1990).

Note that a color composite by either the RGB system or the

IHS transform uses a maximum of only three spectral channels

and successful results often depend on a tedious trial-and-

error process.

Daily et al. (1979) were among the first to recognize the

importance of multiresolution processing of satellite data for

remote sensing applications. An airborne radar image with a

10-m spatial resolution was co-registered with Landsat 80-m

MSS images in an effort to improve geological interpretation

for a desert environment. The superior textural variations of

the radar image along with a 10-m spatial resolution were

utilized to complement the low contrast as well as the low

spatial resolution of Landsat MSS data which, in turn,

compensated for the drawbacks of radar shadows. Through a

direct RGB color display method, color composites generated

from the co-registered dataset were able to delineate subtle







22

geologic units through a visual image interpretation. Using

a similar approach, Wong and Orth (1980) also generated useful

color composites from Seasat synthetic aperture radar (SAR)

and Landsat MSS images which have 40-m and 80-m spatial

resolutions, respectively. These two early studies underlined

the benefits in the unified use of satellite data acquired by

completely different sensing systems multispectrall vs. radar)

for improving the interpretability of remote sensing images.

When the RGB color display system is used, satellite

images do not readily define or fit into the red, green, and

blue primary colors (Harris et al., 1990). In other words,

the images can not simply substitute the red, green, and blue

primaries in the RGB display system because of the spectral

incompatibility which could lead to serious color distortions

and poor-quality composites (Haydn et al., 1982; Harris et

al., 1990; and Carper et al., 1990). As a result, the IHS

color perception system has become the widely adopted approach

to resolve the color distortion problems encountered in the

RGB display of multispectral images. The entire process of an

IHS transform for multiresolution processing takes four steps

which include (i) co-registration of multiresolution images,

(ii) a forward transformation from three multispectral images

to the three IHS components, (iii) a reverse transformation

from the IHS components to RGB values, usually with the

replacement of the intensity component by the high spatial

resolution image, and (iv) display the results through a RGB








23

color display system. For the IHS system, the intensity or

brightness of a scene is a function of illumination (Boynton,

1979). Therefore, the intensity component should encompass a

broader range of wavelength (Haydn et al, 1982) and is

extensively associated with the spatial relations of scene

objects (Judd and Wyszechi, 1975). For this reason, the

intensity component is always assumed to be replaced by the

high spatial resolution panchromatic image in the reverse IHS

transformation.

Zobrist et al. (1979) were among the first to apply the

IHS transform to satellite data for image enhancement. In the

study, Landsat MSS 80-m and meteorological Seasat 25-m radar

images were used. The intensity component transformed from

the Landsat MSS data was simply replaced by the 25-m radar

image. Then, an IHS reverse transformation was taken to

create color composites.

Haydn et al. (1982) further demonstrated the utility of

the IHS transform for image enhancement. Landsat MSS, Landsat

return beam vidicon (RBV), and the Heat Capacity Mapping

Mission (HCMM) thermal infrared (TIR) images with respective

spatial resolutions of 80 m, 30 m, and 600 m were merged

between the RBV and MSS and between the MSS and HCMM images.

A direct replacement of the transformed intensity component by

the corresponding high spatial resolution image was employed

in the reverse transformation for each case. Also, ratioed

data between spectral wavebands was demonstrated for the use








24

of the IHS transform. For example, while the intensity

component was transformed from Landsat MSS wavebands four,

five, and seven (denoted respectively as MSS4, MSS5, and

MSS7), the H and S components were substituted, respectively,

by the MSS5/MSS4 and MSS5/MSS6 ratioed data. Substantial

enhancement in color composites was observed. The IHS color

transform was adopted in the entire study because the direct

RGB color model produced confusing and low quality image

presentations. Using a similar methodology, Welch and Ehlers

(1987) were able to produce enhanced color composites from

Landsat (30 m) thematic mapper (TM) and SPOT HRV 10-m

panchromatic images.

Very different approaches for using the IHS transform

have also been reported. A color composite was created from

a Seasat mono-band radar image (Daily, 1983). Both the strong

and weak radar responses related to sloping targets and

vegetation features were extracted, respectively, by high-pass

and low-pass filters, and then used as the hue and saturation

components while the original radar image was used directly as

the intensity component for the IHS transform. The color

composite was able to reveal major structural features

invisible in the original black-and-white radar image. Harris

et al. (1990) took a step further when combining Landsat 30-m

TM and 10-m airborne radar images. In two instances, the high

spatial resolution image was used directly as the intensity

component, while the hue components were created from a








25

combination of Landsat TM wavebands two, four, and seven

(denoted as TM2, TM4, and TM7) or of Landsat TM wavebands two,

five, and seven (denoted as TM2, TM5, and TM7). However, the

saturation component was held at a constant value (150). In

another instance of the same study by Harris et al. (1990),

the radar image was used as the intensity component and the

geological units (numerical codes) in a digitized map as the

hue component, while the saturation component was held at a

constant (150). Based on a visual assessment, the study

concluded that the color composites were able to define

lithological and structural features that were absent from

existing geological maps. These two studies by Daily (1983)

and Harris et al. (1990) not only opened a new dimension in

the use of the IHS transform for remote sensing applications,

but also demonstrated the effectiveness and compatibility of

the IHS transform for a broad range of data characteristics

including satellite images and digital maps.

The use of the IHS transform can be extended to include

the imagery digitized from an aerial color infrared (ACIR)

photography (Grasso, 1993). In the study, a digitized ACIR

high spatial resolution (10 m) image was merged with Landsat

MSS and Landsat TM data to enhance geological interpretation.

The intensity components transformed from either Landsat MSS

or Landsat TM images were directly replaced by the ACIR image

during the IHS reverse transformation. The color composites,

which had a 8x linear spatial resolution factor, were still








26

useful for geological mapping. Also in the study, a different

approach in utilizing the IHS transform was demonstrated in

which the digitized ACIR image was used as the intensity

component, the Landsat TM ratio data (TM5/TM7) as the

saturation component while the hue component was held at a

constant value (96). The results were very useful for

delineating the high and low clay content areas. Note that

the high and low TM5/TM7 ratios were essentially used to

regulate the level of color saturation (S component) so that

high clay content areas would show more vivid colors than its

counterparts. However, results also showed that the colors of

these composites could change very rapidly by just varying the

hue component with a moderate magnitude. Though the concept

of the latter example is somewhat different from the previous

one by Harris et al. (1990) who emphasized the color diversity

(H component) rather than the color purity (S component), good

quality color composites can still be produced by the IHS

transform. This indicates that the IHS color transform has

tremendous flexibilities in adapting to a wide variety of

geographic data.

In summary, when applying the RGB and IHS methods to

generate color composites, it has been demonstrated that image

interpretability can be significantly improved through a

unified use of multiresolution datasets. This is particularly

evident for the IHS transform which is capable of producing

quality color composites under a broad range of circumstances.








27

Also, several studies have illustrated that the IHS transform

seems to possess a virtually universal adaptability to

geographic data. The multiresolution merged results in the

form of color composites are indispensable for many remote

sensing applications which involve image interpretation.

However, to generate color composites is not the ultimate goal

of multiresolution processing of satellite imagery data. The

radiometric quality of merged images is far more important

than a color display and vital to the potential of post-

merging applications such as land-use classification. In

addition, a severe disadvantage for color composites is that

tremendous efforts are needed to extract quantitative land-use

information from the color products while a maximum of only

three spectral wavebands can be handled at one time.


Radiometric Enhancement


In the radiometric enhancement approach, arithmetical

algorithms are used to digitally combine the multiresolution

image data in order to generate merged images which can

achieve the purpose of multiresolution processing. Direct

substitution of the high spatial resolution image for the RGB

color display system often created color composites with

blocky appearances because of the spatial resolution

differences. To overcome such a weakness, digital

manipulations of the image data have become necessary. In a

study by Cliche et al. (1985), simulated SPOT HRV panchromatic







28

and multispectral images from an airborne dataset, which had

11 spectral wavebands, were digitally merged, pixel by pixel,

using the following methods


I. MIi = Ai (PAN HRVi)W + Bi [3-1]


II. MIi = Ai (PAN HRV,) + Bi [3-2]


III. MI1 = Al (PAN HRV,)1 + B, [3-3a]

MI2 = A2 (PAN HRV2) 2 + B2 [3-3b]

MI3 = A3 (0.25PAN + 0.75HRV3) + B3 [3-3c]


where MIi is the merged multispectral images, i (in methods I

and II and subscripts 1-3 in method III) is waveband index,

PAN and HRV are, respectively, the simulated SPOT panchromatic

and multispectral images, and Ai and B, are coefficients or

scaling factors to maintain the merged data within the 0-255

dynamic range.

From the color composites generated through the use of

the RGB color display system, the study concluded that, while

the improvement on spatial resolution was apparent for all

three methods, method (III) produced the best color composite.

The improvement by method (III) was attributed to the use of

different merging algorithms which helped preserve the SPOT

HRV3 near infrared information. For method (II), the pixel

values were low and concentrated, resulting in dark and no-

contrast merged images. Because of the high correlations in

the merged images between the near infrared and visible








29

wavebands, method (I) produced wash-out images. Even though

all these merging methods were based on arbitrary speculations

and the results were displayed using the RGB system, the

potential benefits of digitally merging image datasets were

indicated with improved spatial information.

In digitally merging multiresolution images, speculations

for a combining approach do not bring about consistent

results. In an effort to find a general approach that does

not depend on arbitrary elaborations, Price (1987) contended

that the high correlations between the panchromatic and both

the multispectral green and red wavebands within a SPOT

multiresolution dataset could be utilized to estimate the

corresponding high spatial resolution multispectral merged

images. In the study, the original SPOT 10-m panchromatic and

20-m multispectral images were artificially degraded, by

averaging, to 20-m panchromatic (P20) and 40-m multispectral

(M40) images, respectively. Then, the whole approach took two
steps. The 20-m multispectral merged images (MI) were first

estimated from the degraded panchromatic P20 data by a

regression equation


MIi = Ai P20 + Bi + D [3-4]


where MIi is the estimated multispectral image i based on the

degraded (20-m) panchromatic image, A, and Bi are regression

coefficients determined from the degraded panchromatic P20 and

the original 20-m spatial resolution multispectral image, and








30

D is a correction factor to balance the numerical sum of

estimated subpixels with the recorded value of a low

resolution pixel (M40). If the sum of the estimated digital

counts of subpixels did not equal the recorded value of the

low resolution pixel in question, a correction was applied.

The estimated 20-m images from the degraded (40-m)

panchromatic waveband were able to retain 99% of the variances

of the original 20-m images of the green and red wavebands.

However, a potentially serious problem could have existed with

a high correlation between the two estimated images because

they both depended on the same identical panchromatic data.

In fact, the multispectral images were used only as

complementary information through a correction procedure.

A different approach was undertaken to estimate the SPOT

near-infrared (NIR) image which in general does not correlate

well with the panchromatic waveband. The (estimated) merged

20-m NIR image (MI3) was first obtained from a lookup table

created by both the degraded 20-m panchromatic (P20) and the

original 20-m NIR images. Then correction was applied similar

to those used for the green and red wavebands. Results

indicated that only 75% of the radiometric variance of the

original NIR image was retained during the merging process.

Though the broad spectral bandwidth of the panchromatic image

encompasses part or even the entire range of the multispectral







31

green and red wavebands, it is impossible that the portion of

image digital count for a merged high spatial resolution image

can be separated from a panchromatic pixel. The difficulty is

analogous to isolating from a jar of oil the part that came

from a particular peanut. In addition, the process of spatial

degradation (by averaging pixels) could smooth out or compress

the radiometric information in the original image data.

To explore the utility of digital manipulations for

datasets acquired by multiple sensors, Landsat MSS and Shuttle

imaging radar A-band (SIR-A) images were digitally merged in

a lithological mapping study (Chavez et al., 1983). However,

the high spatial resolution of the 40-m radar image was not

utilized to its advantage for enhancing the spatial resolution

of the Landsat 80-m MSS images. Instead, the spatial

resolution of the radar image was artificially degraded for

compatibility with that of the Landsat MSS data. From the

results of various arithmetical manipulations including

addition, subtraction, ratioing, and difference-ratioing of

the co-registered SIR-A and Landsat MSS image data, it was

concluded that the addition and ratioing methods were useful

for discriminating some geologic units while the applicability

of the subtraction method is limited only to negatively

correlated images. The results by Chavez et al. (1983) have

two important implications. First, digital manipulations of

image data can be extended to include those images acquired by

different sensing systems. Second, the arithmetical








32

manipulations of image data can be applied to images with

different spatial resolutions as well as those which have the

same spatial resolution.

Digital merging multiresolution images has been used in

efforts to further enhance the results of an IHS transform.

In order to more effectively use the IHS transform, the

selection of a proper intensity or brightness component is

very critical to the quality of color display (Boynton, 1979).

A low intensity component could result in severe image

degradations (Judd and Wyszechi, 1975; Haydn et al., 1982).

In the case of a low intensity value, corrections are needed

for the hue and saturation components (Judd and Wyszechi,

1975) or for the intensity component (Boynton, 1979; Haydn et

al., 1982; Gillespie et al., 1986). However, by applying such

correction procedures, the final image is very difficult to

interpret because the original colors can be altered

significantly (Zobrist et al., 1979).

The importance of finding the most effective method to

generate the intensity component for the IHS transform for

merging multiresolution datasets has been recognized by some

researchers, including Carper et al. (1990). Instead of

adopting the direct replacement of the panchromatic image for

the intensity component, Carper et al. (1990) conducted some

experiments on different merging methods in order to find the

best intensity component. In addition to many previous

studies that relied on imagery data acquired on different








33

dates or even in different years, simultaneously-acquired SPOT

10-m panchromatic and 20-m multispectral images were used.

This was done to eliminate the contribution of temporal

information which could introduce some difficulty to the

assessment of the benefits of an IHS transform. Carper et al.

(1990) proposed the following set of merging algorithms to

calculate the intensity components.


Ia = (PAN + RHV3)/2 [3-5]

Ib = (PAN PAN HRV3)1/3 [3-6]


Ic = (2 PAN + HRV3)/3 [3-7]


Id = (PAN HRV) 1/2 [3-8]

I0 = (HRV, + HRV2 + HRV3)/3 [3-9]

where I, with alphabetical subscripts for method index, is the

calculated intensity component to replace the original

intensity component Io transformed from the 20-m multispectral

images, PAN is the SPOT panchromatic image, and HRV is the

multispectral data with numerical subscripts for waveband

index. The study concluded that the weighted average method

(Ic) consistently produced results as good as or better than
the others. The effectiveness of this weighted average method

(Ic) was attributed to the greater histogram similarity
between the calculated (Ic) and the original (I) intensity

components. However, some points in the results were left








34

undiscussed. For instance, while the histogram of Ic

correlated extremely well to that of the panchromatic image

except with a moderate shift to higher values, it did not have

any resemblance to that of the original HRV3 image. This

indicated that the coefficient (2/3) for the PAN image in

equation [3-7] significantly exaggerated the effect of the

panchromatic image in the Ic component, implying not only a

duplication of the panchromatic information, but also a

significant loss of radiometric information for the HRV3 image

in the merging process. In addition, the great similarity

between the histograms of intensity Ic and the panchromatic

image suggested that a direct replacement of the intensity

component (Io) by the panchromatic image, as used in many

other studies, is workable in an IHS transform.

In response to a broad array of diverse approaches which

have been used to merge multiresolution datasets, several

methods to combine multiresolution images were evaluated by

Chavez et al. (1991) using statistical, visual, and graphical

comparisons. More specifically, those different combining

methods included the IHS transform, the PCA method, and the

high-pass (spatial) filtering (HPF). For the Landsat TM and

SPOT panchromatic datasets used in the study, a contrast-

stretching procedure was applied to the SPOT panchromatic

image in an attempt to increase (arbitrarily scale up) the

radiometric variance. Then, in the IHS method, the intensity

component transformed from Landsat TM images was simply








35

replaced by the contrast-stretched panchromatic image during

the IHS reverse transformation. In the PCA method, the

stretched panchromatic image was assumed to be similar to the

first principal component transformed from the Landsat TM

images of all six wavebands (excluding the TIR waveband),

while in the HPF method, a high-pass filter was applied to the

contrast-stretched panchromatic image to extract the high

frequency spatial information which was merged to each of the

six Landsat TM images through a pixel-by-pixel addition

method.

Color composites generated by all three methods were

subjected to visual comparisons. Statistical correlation

analyses were conducted between the first principal component

of Landsat TM six-waveband data, the IHS intensity component

and the contrast-stretched panchromatic image. Spectral

signatures from five selected land-use types were graphically

compared between the original Landsat TM data and the merged

datasets by the IHS transform and HPF method. Chavez et al.

(1991) concluded that, though the IHS method produced the best

color composite among the three methods, it distorted the

spectral characteristics of the merged images the most. For

the HPF method, the merged images possessed the spectral

characteristics comparable to those of the original Landsat TM

data. The distortion of spectral information by the IHS

method was attributed to the fact that the customary

assumption of similarity between the IHS intensity component








36
and the panchromatic image is not always valid. When one

examines the implicit spectral requirements (in a decreasing

order of spectral bandwidth for the I, H, and S components) by

the IHS transform as discussed by Haydn et al. (1982), it is

not surprising to recognize that the distortions of spectral

integrity would be inevitable in the transformed I, H, and S

components. Note that the requirement for decreasing spectral

bandwidths for the I, H, and S components would generally

result in the numerical values of those components being in

the same order. In using the IHS transformed data for post-

merging applications other than color composites, these

distortions of spectral information cause a serious concern

about the utility and effectiveness of a merged dataset for

multispectral analyses.


Other Enhancement Methods


There were some other cases in which multiresolution

merging was used for purposes other than image enhancement.

It is worthwhile to discuss these methods because of their

pertinence to the subject of merging multiresolution datasets.

The practical importance of digitally merging multiresolution

datasets for image data compression purpose was investigated

by Schowengerdt (1980). He contended that the data volume for

storage and transmission can be significantly reduced if a

high spatial resolution multispectral dataset can be

constructed by combining a high spatial resolution image with








37

a relatively low spatial resolution multispectral dataset.

With that argument in mind, the spatial resolution of the

Landsat MSS images of wavebands four (green), six (NIR), and

seven (NIR) with the original 80-m spatial resolution was

artificially degraded by a linear factor of three to a 240-m

spatial resolution dataset. The original 80-m resolution

image of waveband five (red) remained unchanged and was used

as the high spatial resolution image. Assuming that an image

consists of both spectral and spatial components, the

following merging equations were proposed


MIi = MSSi + ki H5 [3-10]


and


ki = ac / as [3-11]


where MIi is the reconstructed image, i (and subscript 5) is

waveband index, H5 is the high frequency spatial information,

and a is the image-wide standard deviation. The high

frequency spatial component (Hs) was obtained by a subtraction

between the low-pass and the high-pass filtered images of

waveband five. New images with a 80-m spatial resolution were

reconstructed through pixel-by-pixel manipulations using

equations [3-10] and [3-11]. Visual evaluation of the

reconstructed images indicated that a great deal of high

frequency information (edges) could be restored except for

vegetation-dominated areas where a reverse tonal appearance








38
was indicated. Waveband five was selected as the high spatial

resolution image because it had the greatest contrast. This

selection of waveband five would make the ki values by

equation [3-11] smaller than 1.0 because a5 is the largest.

Consequently, equation [3-10] implicitly emphasizes the

multispectral images, making it possible for the merged

datasets to maintain the spectral characteristics of the

original multispectral data.

The utility of an IHS transform for image data

compression was also studied by Haydn et al. (1982) using

Landsat MSS wavebands four (green), five (red), and seven

(NIR). The hue and saturation components transformed from the

three Landsat MSS images were each arbitrarily degraded. The

spatial resolution was reduced by linear factors of two, four,

and six which corresponded to data compression factors of

four, sixteen, and thirty-six, respectively. Color composites

were regenerated for each data compression factor using the

degraded H and S components along with the original I

component. Visual comparisons of the regenerated color

composites to that of the three original wavebands did not

indicate substantial quality deterioration except for the case

which had a data compression factor of thirty-six or a 6x

linear resolution factor.

A half-pixel shifting method to improve the effective

spatial resolution of remote sensing data was studied by Dye

and Wood (1989). They argued that, for a given pixel in a








39
scene imaged twice over a time period, both its numerical

value and geographic location would not be identical because

of the potential offset (error) in sampling the pixel by the

sensor. Therefore, if one of the two images in a dataset is

artificially offset half a pixel before the two images are

combined together, the resultant merged image will increase

its spatial resolution by a linear factor of two. From the

viewpoint of data sampling technique, this method is very

interesting. However, in the remote sensing monitoring of

land-use activities, the concept may not be valid or even

logical when considering the time lapse in image acquisition

and the spectrally dynamic changes in natural environments.

In a study using artificial as well as satellite images,

Albertz and Zelianeos (1990) pointed out the following

requirements necessary for this half-pixel shifting method to

be successful: (1) the scene must be imaged several times--

preferably more than four; (2) there will be no significant

changes in the scene environments (spectrally static objects);

and (3) image geo-referencing or co-registration must be very

accurate in order to have the precise half-pixel offset.

With the advent of geographic information systems (GIS)

techniques, various types of geographical data including

existing map data and multi-date imagery data have been

integrated during an image processing scheme. However, the

main purpose of such image processing efforts is to detect

changes rather than to improve the spatial and radiometric








40

qualities of the final results. To improve agricultural land-

use classification, Lo et al. (1986) combined two Landsat MSS

scenes acquired in different growing seasons. The two scenes

were co-registered and some waveband ratioing was undertaken

before invoking land-use classification procedures. Using

this multitemporal approach, the land-use classification by an

unsupervised classification scheme was improved from 84% to

86%. However, it is arguable that the information accumulated

from the two scenes and the use of more spectral wavebands

would definitely be a factor contributing to the improvement

of classification results. A similar study for corn-soybean

field classifications was conducted by Badhwar et al. (1982)

using Landsat MSS data.

There are many other examples that involved the use of

satellite imagery data and thematic overlay techniques. For

instance, the study by Walsh et al. (1990) combined Landsat TM

images with digital elevation model (DEM) data to study the

hydrological processes in rugged terrain environments, and

that by Shih (1988) who combined Landsat MSS data with the

digitized version of the United States Geological Survey

(USGS) land use/land cover maps within a GIS environment for

land-use classification comparisons.


Summary: Assessment of Problems


Many studies have been made to develop image processing

techniques to combine multiresolution images for remote








41

sensing applications. Opportunities exist to improve the

interpretability of satellite image datasets for the

management and monitoring of natural resources and the

environment. In summary, these efforts have demonstrated the

following aspects.

1. The IHS transform is a powerful and effective method

for generating true color composites of good quality under a

broad range of data characteristics. The effectiveness of the

IHS transform has indicated a virtually universal adaptability

to any geographic datasets. As compared to the direct RGB

color display system, the IHS transform is superior because it

can overcome the incompatibility of spectral information

content of satellite multispectral images. This makes the IHS

transform more likely to produce well balanced color

composites that are more suitable for image interpretation.

However, the effectiveness of the IHS transform has misled

many to believe that it is a powerful image processing

technique that can actually sharpen the image data.

Unfortunately, it is not. The process is only for the display

of colors for human aesthetic pleasure. The merged images by

the IHS transform are not useful for multispectral analyses

because of the inferior radiometric quality and corrupted

spectral integrity.

2. Many combining methods have been developed that vary

significantly in the basic principle as well as in the

complexity of merging algorithms. These methods can be








42

categorized as: (1) linear combination of images, (2)

principal component analysis, (3) regression technique which

is similar to linear combination, and (4) multiplication or

product (including square-root of product). Though arbitrary

and largely dependent on speculation, these methods provide

knowledge about merging multiresolution images. The studies

by Schowengerdt (1980), Cliche et al. (1985), Price (1987),

Carper et al. (1990), and Chavez et al. (1991) suggest that

combining multiresolution images by linear combination of

images would have a greater potential for multiresolution

processing. The multiplication and principal component

analysis methods are perceived as ineffective.

3. The lack of understanding of the principle of

multiresolution processing is ubiquitous, resulting in wide

speculation for merging algorithms. The fundamental problem

is that the effects of combining multiresolution images on the

radiometric, spatial, and spectral qualities of a merged

dataset were not well understood when a merging algorithm was

introduced. Frequently, efforts resulted in radiometrically

inferior and spectrally corrupted merged datasets. A good

merger should take full advantage of the spatial and spectral

benefits of the multiresolution images to create a merged

dataset.

4. The main attention of research efforts was given to

image color display rather than to the radiometric and spatial

enhancement and, the spectral integrity of merged datasets.








43

In remote sensing applications, achieving the best color

display is necessary and often very useful for many

applications, but it is not the ultimate nor the only goal of

combining multiresolution datasets. Instead, the merged

images should be sharpened radiometrically while the spectral

integrity is preserved to enhance the utility of merged

datasets.

5. Visual assessment, which is necessary for evaluating

the quality of color composites, is adopted in most cases as

the only technique for determining the qualities of merged

datasets. However, the subjectivity and great variability of

the technique make many of the efforts inconclusive.

6. One other problem not discussed in the literature is

the accuracy of image co-registration. In order to merge

multiresolution images correctly, an accurate co-registration

is required, particularly for high spatial resolution datasets

as well as for images which are not taken simultaneously. For

instance, if two images are not co-registered accurately, a

pixel of one land-use type will be merged with a different

land-use type and the merged pixel belongs to neither of the

original land-use elements. This makes the merged dataset

very difficult or even impossible to interpret and analyze.

Therefore, both the precision of intermediate references (e.g.

maps) and the methods of entering (or digitizing) reference

coordinates must also be addressed (ASPRS, 1990; Bolstad et

al, 1990; and Tan and Shih 1991b). For the current map







44

standard, which is 0.5 mm (1/47 inch) times the reciprocal of

map scale (APSRS, 1990; Bolstad et al., 1991), the

geographical error for the USGS 7.5 minute series maps

(1:24,000) is about 13 m and the digitizing process could

introduce additional errors of significant magnitude (Tan and

Shih, 1991b). Therefore, it will be necessary to utilize

high-precision techniques such as the global positioning

system (GPS) to bypass the intermediate reference (map) as

well as manual digitizing operations in order to achieve a

high accuracy registration or to use datasets acquired by a

satellite sensor equipped with onboard co-registration

capability.

7. To merge multi-date images creates another problem in

evaluating the techniques of multiresolution processing.

Because of the dynamic change of scene environments, it is

difficult to analyze the merged data due to the intermingling

of image spectral information with the temporal effects. This

is particularly important for agricultural lands, as well as

natural environments, because they can change rapidly within

a short period of time. When multi-date scenes are combined,

the merged dataset will naturally contain more information

than any of the original ones. Therefore, it will be

difficult to objectively assess the possible improvement as

well as to evaluate the processing techniques. While the

temporal effects could be used for improving land-use








45

classifications (Badhwar et al. 1982; Lo et al., 1986), it

does create difficulties in evaluating the technique.

Fortunately, future satellite sensor systems can provide

simultaneous multiresolution sensing capabilities as well as

onboard image co-registration techniques (Spotlight, 1991;

EOSAT, 1992a; 1992b). Therefore, the problems with multidate

merging and image co-registration will no longer be a concern

to the user community of future satellite remote sensing data.





CHAPTER 4
PRINCIPLE OF MERGING IMAGES


This chapter is focused on the principle of merging

satellite remote sensing images. After the fundamental

principle is presented and discussed, three merging methods

are examined. However, the demonstrations and discussions of

the effectiveness of the merging methods are provided in

chapter 5 using actual satellite images.


Principle of Merging Images


Merging multiresolution images requires the use of

arithmetical manipulations to digitally combine the image

data. An effective merging approach will take full advantage

of the spectral, spatial, and radiometric merits of the images

to be combined to generate merged image data with enhanced

qualities. To develop successful merging methods for remote

sensing applications of multiresolution image datasets, an

adequate understanding of the fundamental principle for

digital manipulations of image data is essential. Therefore,

to assist such efforts in exploring this principle, it is

advantageous to conceptualize remote sensing image data so

that the factors affecting the spatial, radiometric, and

spectral qualities of merged images can be identified,

evaluated, and assessed.










Assumptions


A digital image can be considered as a set of repetitive

digital numbers that are constrained to a spatial arrangement

which is determined by the relations of objects present in the

scene. In virtually all image processing efforts, this

spatial arrangement is not important because it serves only to

reveal where an object or activity is identified rather than

to indicate how and why the decision is made in the process.

Therefore, a digital image is similar to a random variable.

The numerical values of a remote sensing image, which are

often called digital counts (DC), have a distribution depicted

by the image histogram.

The radiometric variance of an image is an important

indicator of the image radiometric quality, and like a random

variable, it can be assessed by the variance of image data.

For a given scene environment, a larger radiometric variance

indicates that scene activities are recorded in more detail.

Throughout this dissertation, the term "radiometric variance"

will exclusively refer to those image data that have not been

subjected to procedures such as spatial filtering and

contrast-stretching discussed in chapter 3.

The assumption that an image is similar to a random

variable will allow the statistical variation analyses of

random variable manipulations to be applicable to image data.

From previous research efforts by Cliche et al. (1985); Price

(1987); and Carper et al. (1990), the method of linear








48
combination of images was considered to have the greatest

potential for multiresolution processing. Therefore,

attention will be given to these combining methods, which will

include summation and differencing of image data. To better

understand the benefits as well as to assess the drawbacks

from manipulating remote sensing images, the arithmetical

functions of summation and differencing of random variables

for statistical variation analyses will be briefly reviewed.

Such a review is necessary in order to understand the existing

merging techniques as well as to develop new merging methods

so that remote sensing images can be manipulated more

productively. For the purpose of clarity, continuing

discussions will be limited to the circumstance of merging two

random variables or images, though three or more variables can

be manipulated at one time. Also, images with the same

spatial resolution will be examined first before proceeding to

the discussion of multiresolution merging.


Arithmetic of Random Variables


It is necessary to examine the arithmetical functions of

random variables to effectively investigate the various forms

of digital manipulations of image data and to assess the

results of such manipulations. According to Mood et al.

(1974) and Mendenhall et al. (1986), combining (both summing

up and differencing) two random variables X, and X2 with means

I, and U2 and variances a12 and a22, respectively, will create







49

a merged variable (Y) which is expressed in a general form of

Y = a X1 B X2. [4-1]

This new variable Y will have a mean value (AY)


y = a A B 42 [4-2]

and a variance (a 2)

oy = a22 +2 B2 2 a B cov(X1, X2) [4-3]

where a (>0) and B (>0) are numerical constants and cov(X1, X2)

is the covariance between X1 and X2. In digitally combining

images, Y is the merged image, X, and X2 represent images one

and two to be combined, and the corresponding constants a and

B are often called weighting factors or merging coefficients.

The covariance cov(X1, X2) term in equation [4-3] can be

written as (Mendenhall et al., 1986)


cov(X,, X2) = r aC a2 [4-4]

where al and 02 are the standard deviations and r is the

correlation coefficient for X1 and X2. The value of r can be

negative or positive depending on the actual relationship

between variables X, and X2. Substituting the covariance of

equation [4-4] into equation [4-3] will yield

a =a2 a12 + B2 a22 2 a B r 01 a2 [4-5]


which is the equation for calculating/estimating the variance

of a merged variable based on the merging coefficients, the








50

variances (or standard deviations), and the correlation

coefficient for X1 and X2.

For a merged image, the quality factor of greatest

concern is the contrast (or gray shades), and the contrast of

an image is directly related to the variance of image

radiometric data. For instance, an image will have no

contrast if its radiometric variance is zero. Therefore,

attention in the continuing discussion will be given to the

variance (a2) of merged variable Y. From equation [4-5], the

factors that collectively affect the radiometric variance or

contrast of a merged image are the weighting coefficients a

and B, the correlation coefficient (r), and the variances (a12

and o22) of the two images to be combined.

To assist the efforts in examining the effects of these

various factors on the variance (ay2) of merged variable Y, it

would be advantageous to reduce the number of the involved

elements in equation [4-5]. One method to achieve that is to

normalize the variances of X, and X2 to unity (1.0) using the

following equation


f 2 + 222 = 1 [4-6]

where a,2 and .22 are the normalized variances for X, and X,

respectively. If the condition a12+ 220 is satisfied, a2 and

922 are defined, respectively, as


a2
12 [4-7]
a12+ 22










and

2
2 2 = [4-8]
a 2+a 2


For easy comparisons, let a 2 also be normalized to (o02+o 2) by

the following equation


02
2= a [ 4-9]
-Y 02 2


where o2 is the normalized variance of Y. Note that the

normalized values are a relative measure for the variances of

X,, X2, and Y. Dividing equation [4-5] by (012+022) and making

rearrangements through the use of equations [4-6], [4-7], [4-

8], and [4-9] will yield


2 = a2 a12 + 2 (1-a 2)

2 a B r 1a j(1-_al) [4-10a]


Because the variances of X1 and X2 are normalized to unity,

equation [4-10a] can also be written as


l2Z = a2 (1-_,2) + B2 a22

2 a B r o2 J(l-az) [4-10b]


where


= J('-7) [4-11a]


2z = J(-- ) [).


[4-11b]







52

Three benefits result from normalizing the variances of X1 and

X2. These benefits are (1) reduction of the number of the

involved factors in equation [4-5]; (2) relief from getting

involved with actual image data for conceptual discussions;

and (3) easy comparison of the variance of the merged variable

with those of the original variables. It becomes clear that

equation [4-10a] (or [4-10b]) is the basic relation that

reflects the effects of the various factors (a, B, r, and a)

on the radiometric variance or contrast (a2) of a merged

image.

A comparison of the relations between equations [4-1] and

both [4-10a] and [4-10b] reveals that the only distinction

between summation and differencing of two variables is the

sign for the last terms in equations [4-10a] and [4-10b].

Therefore, in the context of evaluating the variance of the

merged variable, differencing two negatively correlated

variables (r<0) is technically identical to summing up two

positively correlated ones (r>0). Because merged image data

must be positive, differencing two images may require the

addition of a positive constant (C) to the end of equation [4-

1] such that


Y = a X, B X2 + C [4-12]


in order to avoid negative image data. However, from the

relations of equation [4-10], the constant C in equation [4-

12], which is usually determined by a pre-merging scanning of







53

the given image data, will not affect the radiometric variance

of the difference image.

In practical applications where two images are given, the

radiometric variances (a 2 and a22) and the correlation

coefficient (r) are known. The only factors that need to be

determined for equation [4-1] are the merging coefficients a

and B. From the relations of equation [4-10] and based on the

given factors a12, a22, and r, the selection of appropriate

merging coefficients a and B for equation [4-1] is the key

factor that affects the radiometric variance of merged image

data. To assess the impacts of these merging coefficients (a

and B) on the radiometric variance of merged images, three

approaches for digitally combining images will be discussed.

In addition, of the two images (X1 and X2) to be combined, X1

will be denoted as the primary image and X2 as the secondary

image in order to distinguish their relative importance in the

merging process. When actual image data are used, the primary

image (X1) will be assumed to contain primary information

while the secondary image (X2) is used as the supplementary

data for improving the primary image.


Confining Method


The first method to be discussed is the confining method

which is defined mathematically as


[4-13]


Yc = a X1 + B X2








54

where YC is the merged image by the confining method, X, and

X2 are, respectively, the primary and secondary images, and a

and B are weighting coefficients. An unique aspect in

combining images is to keep the merged image data within the

0-255 dynamic range (or 8-bit data depth). One approach to

accomplish that requirement is to choose the weighting

coefficients a and B in equation [4-13] such that


a + B = 1 [4-14a]


which can be written alternatively in the following forms of


a = 1 B [4-14b]


and


B = 1 a. [4-14c]


Because the merged image data is automatically confined to the

0-255 dynamic range, this merging method is called the

confining approach.

Let o 2 and o22 denote the normalized radiometric

variances for the primary (X1) and secondary (X2) images,

respectively. Because the general purpose to combine images

is to use the complementary secondary image data for improving

the primary image, the weighting coefficient B for the

secondary image will be of greater interest. For this reason,

the relation of equation [4-14b] is preferred and by

substituting it into equation [4-13], the following relation









is obtained as


YC = (1-B) X, + B X2. [4-15]

In comparing equation [4-15] with the relations between
equations [4-1] and [4-10a] (or [4-10b]), the normalized

variance (a2) of merged image Yc can be estimated by the

following equation

2 = (1-B)2 a12 + B2(1-, 2)
+ 2r(l-B)B a, J(1--0). [4-16a]

Since the variances of X1 and X2 are normalized to unity,

equation [4-16a] can also be written as


gZ = (1-B)2(1-022) + B2 22
+ 2r(1-B)B 02 J/(1-2) [4-16b]

The following relations for the normalization of variances are

also needed in order to use equation [4-16a] or [4-16b]


ai2 + 22 = 1 [4-17]


S1, = T(9) [4-18]


_z = (Z2'). [4-19]

From equations [4-16a] and [4-16b], the radiometric variance

(af2) of an image merged by the confining method is influenced
only by the secondary image coefficient 8. The value B will








56

have a direct impact on _2 -- a measure of the radiometric

quality of a merged image by the confining method. In

combining multiresolution images, selecting an appropriate B

value is particularly important because it not only affects

the radiometric variance, but also indirectly impacts the

spectral information of the entire merged dataset. For

instance, if a large B is used to merge a panchromatic image

to each image in a multi-waveband dataset, all the images in

the resultant merged dataset will be very similar to each

other.

Assuming that the variances of the primary (X,) and

secondary (X2) images are equal or close to each other, the

relation of g2 as a function of weighting factor B is depicted

in Figure 4-1 for the confining method. Although the graphs

in Figure 4-1 can tilt somewhat from one side to the other in

response to the variance difference between the primary and

secondary images, four important observations can be made for

the confining method.

First, the radiometric variance of a merged image by the

confining method is likely to be smaller than that of either

the primary and secondary image data. A smaller radiometric

variance implies that the merged image by the confining method

will have low contrast and inferior radiometric data.

Second, the state (positive or negative) as well as the

strength of correlation between the primary and secondary

image images also has a strong effect on the radiometric










57






I I I ,





OO *
oo


000




41







.4 O 4q.1
C1 m 44
-,-
o 40

+ H



.- 00

\0 0V-




4 O











S IV 0
,o p4o







-,,
o 0 4
\ <0 0
S* 2 go











0 |40









4) 0
0 -I00 I .- 0 0 r-4 0
I f l II I I I









S A o o r) 0c






-H
o E: U








58

variance of a merged image. If the primary image is

negatively correlated to the secondary image, the loss of

radiometric information in the merged image (Yc) will be even

more detrimental as shown in graphs (4) and (5) of Figure 4-1.

The negative correlation (r<0) creates a negating effect on

the variances of the primary image when the secondary image

data is digitally merged. Consequently, the resultant merged

image will have low or even no contrast depending on the

strength of the correlation as well as the use of B values

(Figure 4-1). As mentioned earlier, adding up negatively

correlated images is similar to subtracting positively

correlated ones or vice versa. If two negatively correlated

images are difference instead of summed together as done by

Chavez et al. (1983), the loss of radiometric information in

the merged image can be alleviated. In this circumstance, the

results of graphs (4) and (5) will be changed to those of

graphs (2) and (1) in Figure 4-1, respectively.

Third, a Bc exists at which the radiometric variance of

merged data will be at minimum. That is, the merged image

with Bc value will have least contrast. Therefore, the use of

such a B value must be avoided when using the confining

method. The value of BS can be obtained by first taking the

derivative of equation [4-16a] (or [4-16b]) with respect to B


d (c2)
-= -2(1-B)g,2 + 2B(1-a,2)
dB


+ 2r a1 .(1-a/ ) (1-2B)


[4-20]








59
and then setting the first derivative to equal to zero such as


0 = -2 (1-B)a,2 + 2Bc(1-Ol2)

+ 2ra1 J(1-g4) (1-2B) [4-21]

Through the use of equations [4-7], [4-8], [4-9], and [4-17],

the BS value can be estimated by the following equation


0a, ra, a2
c = [4-22]
C, + 02 -2 r 0, 02


where a,2 and a 2 are the variances (a, and o2 are the standard

deviations) of the primary and secondary images, respectively.

Note that the range of valid values for B is 0 to 1. If Bc is

outside the 0-1 range, the minimum radiometric variance of a

merged image will not exist within that range. Obtaining the

Bc value before merging the images will give a first

assessment on the variance of a merged image. For instance,

if BC 5 0, the radiometric variance of merged data is an

increasing function with B, implying that an improvement for

image contrast is possible. If Bc 1.0, the variance of

merged data will decreases as B increases. As a result, the

contrast of the merged image will deteriorate. By

substituting BS into equation [4-15a] (or [4-15b]) and by

using the relations of equations [4-7], [4-8], [4-9], and [4-

17], the minimum variance (o,2) for an image merged by the

confining method can be estimated as










ol2 a22 (1 r2)
S= [4-23]
o12 + 022 2 r 0 a2


Caution should be exercised in using equation [4-23] to

estimate the minimum radiometric variance of a merged image.

If BC is not within the 0-1 range, the estimated minimum

radiometric variance is a false value that cannot exist for a

merged image.

Fourth, as B continues to increase beyond the Bc value,

the variance of merged image YC is approaching that of the

secondary image data. This will make the merged image more

and more similar or even identical to the secondary image as

a result of large B values, which has been indicated by Carper

et al. (1990). In the case of merging a high resolution

panchromatic image to a set of multispectral images, all the

resultant merged images will be highly correlated among each

other because of the excessively redundant panchromatic data.

Consequently, the spectral integrity (or signatures) of the

merged dataset will be corrupted and the effectiveness of the

merged multispectral data for differentiating land-use

elements will be reduced.

The variance difference between the primary and secondary

images can also have an effect on the radiometric variance of

a merged image as shown in Figure 4-2. For instance, if the

primary image has a larger variance relative to that of the

secondary image, the merged image will have less contrast

regardless of the state of correlation. This is illustrated






































































r-i \ CO b-



p^Bp pabxemu


o 0

0O aOUPT2IA


*o
04










a)
44I








0




.r4
a
0
*4
ra

a,

4J

0
04.4
4)0
c




4 -I
CO






'U
c *

0 0
CO
4 )



eO
'.4


404
SW4


Eal *4


paZTu r-4lN

paT"euHi>








62

by graphs (1) and (3) of Figure 4-2. If the primary image has

a smaller variance, the radiometric information in the merged

image will either increase or decrease depending on the

magnitude of the variance difference as well as the state of

correlation between the two images as shown by graphs (2) and

(4) in Figure 4-2. Note that only when the primary image has

a very small variance relative to that of the secondary image

and the correlation between the two images to be combined is

high and positive, will an image merged by the confining

method have an enhanced contrast. This indicates that (1) the

confining method is not an effective merging approach for

digitally combining images and (2) the determination of a B

value for the confining method can not be arbitrary nor

independent of the factors such as the variance difference and

the correlation between the two combining images.

The way by which the merging coefficients are determined

(a+B=l) for the confining method has one important implication

of the compromising effect on the quality of the primary and

secondary images. The use of a larger B value to emphasize

the effect of the secondary image is made at the concession of

the primary image variance because of a smaller a value. As

shown in Figure 4-2, this concession of the primary image data

can be beneficial or detrimental. If the secondary image has

a relatively larger variance, this compromising effect is

beneficial to improve the primary image as shown by graphs (2)

and (4) in Figure 4-2. On the other hand, the effect will be








63

deleterious to the radiometric quality of a merged image as

illustrated by graphs (1) and (3) in Figure 4-2.

In summary for the confining approach (a+B=l), the

following observations are as follows. (1) The resultant

merged image will likely have a smaller radiometric variance

or lower contrast unless the primary image has a very small

variance relative to that of the secondary image and the

correlation between the two combining images is high and

positive. (2) There may exist a Bc value at which the

radiometric variance or contrast of the merged image will be

minimum, therefore, the selection of B values close or equal

to Bc should be avoided. (3) Two images with a negative

correlation should be difference rather than summed in order

to minimize the loss of radiometric information. (4) In

general, the contrast (or variance) and brightness of an image

merged by the confining method can be considered as a

compromise for each of these two quality factors between the

primary and secondary images.


Preserving Method


For most satellite imagery, the spread of image digital

data does not extend throughout the entire 0-255 dynamic

range. For a typical agricultural scene, the data spread is

about 40% of the 0-255 range in Landsat imagery (Price, 1984)

while a much smaller range is often found for SPOT images.

Therefore, the utility of the 8-bit data depth for these







64

images has not been fully utilized. In merging satellite

images, such a deficiency can be turned into an advantage by

maintaining the primary image unchanged (a=l) while the

secondary image data is merged. Hence, this method is called

the preserving approach.

By using the preserving approach to combine images, the

following merging algorithm is used


Yp = X, + B X2 [4-24]

where B is weighting coefficient, Yp is the merged image, and

X, and X2 have been defined previously. Let g12 and g2 denote

the normalized variances for the primary and secondary images,

respectively. A comparison of the relations between equations

[4-1] and [4-10a] (or [4-10b]) indicates that the normalized

variance ( 2) of an image merged by the preserving method can

be estimated by

S= 2 2+ B2(1-a_2) + 2rB a,1 J(1-Iz). [4-25a]


Since the variances of the primary (X,) and secondary (X2)

images are normalized to unity, equation [4-25a] can also be

written as

fp2 = (1-22) + 2 22 + 2rB a2 J/(1-2 ) [4-25b]


In both equations [4-25a] and [4-25b], B is the weighting

coefficient and r is the correlation coefficient for the

primary (X1) and secondary (X2) images. Note that the variance








65

of merged image Yp is normalized to o +o22. Also, the

relations of equations [4-17], [4-18], and [4-19] are needed

when using equation [4-25a] (or [4-25b]) for assessing the

radiometric variance of a merged image. As mentioned earlier,

the only distinction between summation and differencing of two

images is the sign for the last term of equation [4-25a] (or

[4-25b]). Thus, differencing two negatively correlated images

(r<0) is identical to summing up two positively correlated

ones (r>0).

Figure 4-3 shows the normalized variance g2 of merged

images by the preserving method as a function of both the

merging coefficient (B) and the correlation coefficient (r).

When the primary and secondary images are not negatively

correlated, the variance of an image merged by the preserving

method is an increasing function with merging coefficient B as

shown in graphs (1) through (3) of Figure 4-3. This implies

that the radiometric variance (contrast) of an image merged by

the preserving method will surely improve, provided that the

correlation coefficient is r20. Unlike the confining method,

which tends to make a compromise between the secondary and

secondary images, the preserving method does not subdue

(because a=l) the radiometric variance of the primary image

during the merging process. Consequently, the merged data are

always enhanced even when the images to be combined are not

correlated (r=0) as shown in graph (3) of Figure 4-3.

The results from combining two negatively correlated

images are also shown by graphs (4) and (5) of Figure 4-3.










66









0
O D4



\\ 0\ i^

S\I OB 0,-



o



,0.)
C; 00









up p Bop
S- *Ob'




OO.


S0 4
t 4a
go k






\ \ 4 .
\\ 0 001







\; o I I r.o 4





\qI 00
w-4O-r 0 0



(0 $4
00 ko v N r 00 % I 1 C a








T~C~m~r) rl

N 03 O (V rl 0 \O (O
*^
I-I ( rlr( O O O








67

Apparently, any improvement on the radiometric variance of

merged image Yp is unlikely, particularly when the correlation

coefficient rz-l. However, such negatively correlated images

can be difference. This will reverse the results to those of

summing up two positively correlated images as depicted by

graphs (1) and (2) of Figure 4-3. This differencing approach

will alleviate or even avoid the loss of image contrast in the

merged data.

The radiometric variance difference between the primary

and secondary images also has an effect on the radiometric

variance of a merged image (Figure 4-4). However, the effect

does not cause a negative impact on the radiometric variance

of merged data. The image contrast will always improve, and

the extent of improvement is inversely related to the

magnitude of the variance of the primary image. When the

radiometric variance of the primary image is relatively small,

the improvement on the merged image is more notable as

illustrated by graphs (1) and (3) of Figure 4-4. If the

radiometric variance of the primary image is relatively large,

the radiometric improvement might not be so apparent,

particularly when the correlation between the two combining

images is near zero (rz0) as shown by graph (4) of Figure 4-4.

It needs to be pointed out that large values of B can not

used for the preserving approach. Otherwise, a scaling factor

must be introduced to equation [4-24] in order to keep the

merged image data within the 0-255 range. In this case, the









68





14


0




X


*4
o~



o
44' 0






Sas U
0
N0

0 O




o 4 C






I I I I I I I
C) U 0
0 0 VU


\N \1 \-o r H o 4.







0 \.4 \0 -
S-r4








4r
~ p~reuIJO 8U~TIA p~T~UiO-
5.








69

use of an additional scaling factor will make the preserving

method less effective or even similar to the confining method.

In summary, several observations are made for the

preserving approach: (1) if the correlation coefficient (r)

between the primary and secondary images is non-negative

(rO), the image contrast in the merged data will surely be

enhanced by the preserving method; (2) as compared to that of

the confining method (a+B=l), the effect of the variance

difference between the two combining images will not create a

negative impact on the radiometric variance of a merged image

by the preserving method, provided that the correlation is

non-negative (r > 0); (3) the preserving method, which does

not subdue (a=l) the primary image in the digital merging

process, will make it less likely that the spectral signatures

of the original multispectral dataset will be altered or

corrupted in a merged dataset; (4) two images with a strong

negative correlation (rz-l) should be difference instead of

summed together in order to avoid a potential loss of

radiometric information in the merged image; and (5) the

preserving method has both a much smaller sensitivity to the

strength of correlation and a larger range of B values to use

because a minimum variance does not exist provided that r0O.


Differencing Method


From previous discussions on both the confining and the

preserving methods, it is known that, in order to enhance the







70

radiometric variance of merged image data from negatively

correlated wavebands (r<0), the differencing method must be

used. To ensure that the merged data will be positive, a

constant must be added to the merged data. Therefore, the

following relation will be used as the merging equation for

two negatively correlated images


Yd = X, B X2 + C [4-26]

where B (>0) is a weighting coefficient, C (>0) is a constant

to avoid negative merged data, Yd is the merged differencee)

image, and X1 and X2 have been defined previously.

Let g,2 and a2 denote the normalized variances for the

primary and secondary images, respectively. By comparing

equation [4-26] with the relationship between equations [4-1]

and [4-10a] (or [4-10b]), the normalized radiometric variance

(9-2) of a merged image by equation [4-26] can be estimated by

d2 = (122) + B2 22 2rB ag j(1-2) [4-27a]

Since the variances of primary and secondary images are

normalized to unity, equation [4-27a] can also be written as

a2 = 2 + B2(1-a12) 2rB a1 /(1-1 ). [4-27b]

In both equations [4-27a] and [4-27b], r is the correlation

coefficient for the primary and secondary images and B is a

weighting coefficient. Note that the variance of the merged

image is normalized to the sum of the variances (a2+a 22) of








71

the primary and secondary images. Again, the relations of

normalization equations [4-17], [4-18], and [4-19] are needed

in order to use equation [4-27a] (or [4-27b]) for estimating

the radiometric variance of a pre-differenced image.

Because the last term in equation [4-27a] (or [4-27b]) is

negative and the correlation coefficient (r) is also negative,

the relation of equation [4-27a] (or [4-27b]) is identical to

that of equation [4-25a] (or [4-25b]) of the preserving method

discussed previously. Therefore, additional information for

the effects of B values, correlation coefficient, and variance

difference on the radiometric variance of merged data can be

found in the previous section for the preserving method with

reference to both Figures 4-3 and 4-4.

In order for an image merged by the differencing approach

to have an improved radiometric variance, it is essential that

the last two terms in equation [4-27a] (or [4-27b]) be 2 0.

That is

B2 a2 2rB a2O 2 0. [4-28]


Hence, if both B*0 and 2*20, a critical Bd value for the

differencing method can be obtained as


2 r a
Bd [4-29]
-2

According to the relations of equations [4-17] through [4-19],

equation [4-29] can be rewritten as (a,40)








72

Bd, 2 r al/a2 [4-30]

where a, and 02 are the standard deviations for the primary

(X,) and secondary (X2) images, respectively. If a B value is

greater than Bd, the variance of a merged image by the

differencing method will increase. Otherwise, the merged

differencee) image will have a decreased radiometric

variance.

It must be pointed out that the relative magnitudes of

the combining image data can have a serious impact on the

tonal appearance of the merged image. Assuming that the

primary image (X,) has relatively higher values (brighter)

than the secondary image (X2), a subtraction by B X2 in

equation [4-26] will be less likely to create negative values

in the merged data. Therefore, a small constant is needed for

equation [4-26]. This will maintain the tonal gradations of

the primary image, and as a result the bright areas in the

primary image remain bright while the dark areas remain dark

in the merged data. When the secondary image has relatively

larger values, the component of B X2 in equation 4-26] will not

be small in comparison to the primary image (X1) data. This

will likely create negative values of large magnitude in the

merged data, requiring the use of a large constant in equation

[4-26] to offset these negative values. Consequently, the

areas with low values (e.g. water bodies) will have relatively

large image values in the merged data because of the use of a

large constant in equation [4-26]. This could make these dark








73

areas appear bright in the merged image (Yd), suggesting that

the tonal gradations of the original primary image have been

altered. Thus, it must be cautioned that the differencing

method may invert the merged image.


Summary: Principle of Merging Images


The principle of merging images has been discussed under

the assumption that an image is similar to a random variable

with regard to digital manipulations for statistical variation

analyses. Three fundamental approaches, which include the

confining, the preserving, and the differencing methods, have

been put forth for digitally merging image data.

Understanding these methods for digitally merging images

is essential for manipulating remote sensing data. Such an

understanding will render useful guidelines for evaluating the

existing methods as well as for developing new effective

approaches in future image processing efforts for remote

sensing applications.

When two images are digitally combined, the radiometric

improvements on the merged image will depend on three factors

which include (1) the selection of a merging method or merging

algorithm; (2) the correlation (r) between the two combining

images; and (3) the variance difference (al and az) between the

primary and secondary images. The confining approach should

be avoided because of its ineffectiveness for radiometric

enhancement. Unfortunately, this merging approach is the most








74

widely used method (Cliche et al., 1985; Carper et al., 1990).

The preserving method is recommended for merging positively

correlated images while the differencing approach is for those

with a strong negative correlation. In addition, the image

with a brighter appearance should be chosen as the primary

image (X,) for the differencing method in order to avoid a

potential of altering the tonal appearance in the merged

differencee) image.

A summary for the effectiveness of the three merging

approaches is provided in Table 4-1 for an easy comparison.

Actual satellite images will be utilized in chapter 5 to

demonstrate the results discussed throughout this chapter.










Table 4-1. Summary of the characteristics of different
merging approaches.


Radiometric improvement on merged image

When Confining Preserving Differencing

i : r>0 and

ao >> a2 No Yes No
01 a2 No Yes No
1 << a2 Yes(if r1l) Yes No

ii: r<0 and

a >> a2 No No Yes
a, z ag No No Yes
a1 << a2 No No Yes


Note: r
al
02


correlation coefficient.
standard deviation of primary image.
standard deviation of secondary image.












CHAPTER 5
DEMONSTRATION OF MERGING METHODS


The main objective in this chapter was to verify and

demonstrate the results of the three merging methods discussed

in chapter 4. To begin the process, the radiometric variance

and mean brightness of merged images by the three methods were

examined using the results from an actual satellite dataset.

The visual appearance in both image contrast and brightness

for the merged images were also evaluated.


Satellite Image Data


A satellite scene by the advanced very high resolution

radiometer (AVHRR) on a National Oceanographic and Atmospheric

Administration (NOAA) series satellite was acquired for this

demonstration. The satellite scene had five images recorded

at the 14:05h U.S. eastern standard time on December 14, 1989

by the AVHRR sensor onboard the NOAA-11 satellite. The NOAA

satellite scene consisted of two reflective (red and near-

infrared or NIR) and three thermal infrared (TIR) spectral

wavebands with wavelength characteristics shown in Table 5-1

(Kidwell, 1991). The scene had a local area coverage (LAC) of

the entire south-eastern region of the United States and all

the images of the five spectral wavebands have the same

spatial resolution of about 1,000 m (Kidwell, 1991).








77

Table 5-1. Wavelength characteristics of NOAA-11 AVHRR LAC
images.


Spatial
Waveband# Wavelength range (pm) resolution


1a 0.58 0.68 (red) 1000 m

2a 0.725 1.10 (NIRb) 1000 m

3 3.55 3.93 (TIRC) 1000 m

4 10.30 11.30 (TIRc) 1000 m

5 11.50 12.50 (TIRC) 1000 m

Source: Kidwell, 1991.


a used in this study.
b near infrared.
c -- thermal infrared.








78

The original NOAA-11 LAC scene contained image data in a

10-bit data-depth format where every three pixels were packed

to a 32-bit word (Kidwell, 1991). A program, which runs on a

PC computer environment, was developed (Appendix C) to unpack

as well as to rescale (linearly) these 10-bit data to a 8-bit

data format for compatibility with PC-based image processing

systems as well as display device. For this research, the LAC

scene was clipped to the region of the Florida peninsula

(Figure 5-1) and only the red and NIR images of the clipped

scene were used. The images of the TIR wavebands were

excluded to avoid confusions from merging thermal data. In

the discussions that follow, a NOAA-11 LAC image is simply

referred to the clipped data unless otherwise stated. The

main usage of this clipped LAC scene was for the verifications

of the three different merging methods discussed in chapter 4.

For ease of explanation, the red waveband was arbitrarily

named as LAC1 while the NIR waveband was denoted as LAC2. The

mean, standard deviation, normalized variance, maximum, and

minimum values of the LAC1 (red waveband) and LAC2 (NIR

waveband) images are presented in Table 5-2. The two selected

LAC images were positively correlated with a correlation

coefficient (r) of 0.577.

Because of the noted difference in the radiometric

variances between the two LAC images (Table 5-2), the use of

LAC1 and LAC2 for the primary and secondary images was

alternated for each of the three merging methods. In case I,



















































Figure 5-1. Location of clipped NOAA-11 AVHRR LAC images.








80

Tables 5-2. Standard deviation (a), normalized variance (g2),
mean (A), maximum and minimum values of NOAA-11
AVHRR LAC images.


Waveband A a a2 max min

LAC1 20.54 5.108 0.2286 138 14
(0.58-0.68 Am)

LAC2 26.35 9.383 0.7714 125 11
(0.725-1.10 Am)








81

LAC2 was used the primary image and LAC1 as the secondary

image, and in case II, LAC1 and LAC2 were used, respectively,

as the primary and secondary images. The purpose of this

alternative use for the LAC1 and LAC2 images was to assess the

effect of variance difference between the combining images on

the radiometric variance of merged data. In the differencing

method, the constants (C) used in equation [4-26] are provided

in Table 5-3 for both case I and case II. These constant

values, which were determined by a scanning of the original

image data, were used to avoid negative merged image data in

the difference LAC images for the corresponding B values.


Variance of Merged LAC Images


The normalized radiometric variances of merged LAC images

by the three methods are presented in Figures 5-2 (case I) and

5-3 (case II). In addition, the mean values (brightness) of

these merged LAC images are presented in Figures 5-4 and 5-5

for case I and case II, respectively. The points in these

four figures are the results computed from the actual merged

image data, while the lines represent the estimates obtained

through the equations in chapter 4. While the estimates of

radiometric variance were obtained through equations [4-15],

[4-25], and [4-27] along with the normalized variances (Table

5-2) and a correlation coefficient (r) of 0.577, the mean

digital count (Figures 5-4 and 5-5) were estimated using

equations [4-13], [4-24], and [4-26] for each corresponding










Table 5-3. Offset constant (C) used in the differencing
method for merging LAC images.


B value

Case 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0


I 3 7 11 15 20 24

II 3 5 8 10 13










83






4)0
U


-H
4





o 4- 1 4


w TC Vi
o U o


Omo r. -A-
4 0









r. 4)H
4- 0 4 O











n -M I 0 1 )
>(C 4- 4 1 4






0. t u 0-
-,- 0 SH
-B $ a .
S0 *- 0 0 0 -' I
So o mo i0
\cQ ** C -0 c-r -
\ -H> IOU

tP V r ri $^4.

-. k CO) t c
0 0 o CO



00 0


0P C 0 00
*H E O tfl

Sty
I Cd M ,C1 *



0 0 0


* ** oa

-04
'-4 1-4 0 0 0 0 0 I
4 ) in
~n l U9
~C( IC
*<-
cr ( rl 3 1 ~sr CV









84




0







C;C



V--4 4V
,-I
r.*4









I I O, .
4)






4 00Q
O 0






0. to 0 0%
44 C )






o p .)4%.4 4t













0
\M 4-

O 0 01 4)
C%0 0 0*
U\ V O H
Q r. V 0 U4-'
L q O 0 *





0 n S.0
0 c
0- tp a


00 u C 0

S> I









HOv *H 00
\ \f UCM 3-W











S* I I *4




0 0-400 II*
rl r \ \ ** O O D


o n 60 0 P 0



*p N r a lo o t c

*o ~ e T~ **
^ ^ 0 0 0 0 ^
B^Bp e~l3IT J aOUT~eApaZT[BUI:ON <|
re










85











I*-

4 4 4

*0 .
o J *0

\o rro



4-,





*4 0 0
0 U H0


U) 1 I I w- I 4
\ I QC B


-C 4
O *M-l






\ C oq i to0k
00 0 -- *(



0 0I I
V -- I 00









-H
4 0 0 4)
0 7 0 A
ONQ *0 C -4 $4


ri C0 00
*) 04 O 4
0 U. 0 LO 0 C 0 0 0
al v I I 0 0 ,-
>r\ 0 41 In













quflD Tv4T TP GUx 'aI0
z



( i0 \ Id / O











fr7

















0
0

4

*- .


o U .4 -


0 0q



0- < 0 v .

Sg .4 ,
8 0

i ..I I iiI
\ U C ) e H
0(a 0




>C 1 )-,4
t0

o C .
ci >- O



r4 Ao t m *
0 *-l D* 4 a

0\ CQ II 0W

* \-o vu mm-

2-iL *T 9 !i1 In I
0 0 *4 00

Q*M \ O W r



- O \ 00 0


-o
La ,' \4 ri (*(i B O- .-


a) I
7unoo T 1T6fTP feiuxi 1 .
0 0

-9-








87

merging method. The letters P, C, and D in Figures 5-2

through 5-5 denote the preserving method, the confining

method, and the differencing method, respectively. From the

results shown in Figures 5-2 through 5-5 for all the three

methods and B values used, four observations are in order.

First, the principle of statistical variation analyses

for combining random variables can be applied in assessing the

radiometric quality (variance and brightness) of a pre-merged

image. This provides the basis for understanding the various

forms of digital manipulations of satellite image data and for

assessing the effectiveness of an image processing effort in

remote sensing applications. In practical applications when

two images are given, the values of correlation, radiometric

variance, and mean digital (brightness) for the images to be

merged are known. Therefore, the overall quality in both

radiometric variance (contrast) and brightness of a merged

image can be evaluated based on the merging method and

coefficient (B). This pre-merging evaluation will lead to

more efficient approaches because unproductive efforts can be

eliminated.

Second, an image contains many subvariables representing

the various land-use types distributed throughout the entire

scene. Note that the subvariables do not usually possess the

same correlation, radiometric variances, and mean data values

in the images of a multispectral dataset. When a merging

algorithm is used to digitally combine the entire images, the