UFDC Home  Search all Groups  UF Institutional Repository  UF Institutional Repository  UF Theses & Dissertations  Vendor Digitized Files   Help 
Material Information
Subjects
Notes
Record Information

Full Text 
FINDING DISCRETE REPRESENTATIONS OF THE EFFICIENT SET IN MULTIPLE OBJECTIVE MATHEMATICAL PROGRAMMING: THEORY AND METHODS By SERPIL SAYIN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 1992 Copyright 1992 by Serpil Sayin To my parents, Lutfiye and Huseyin Sayin ACKNOWLEDGMENTS I would like to express my gratitude to my advisor Dr. Harold P. Benson for the continuous guidance and assistance he provided on this project. His constructive remarks have been a source of improvement in my academical formation. I am grateful to the other members of my supervisory committee, Dr. S. Selcuk Erenguc, Dr. Gary J. Koehler, and Dr. Chung Yee Lee for their commitment. I wish to thank Dr. Anthal Majthay for his words of wisdom at times of perplexity. I am grateful to many friends who helped make my stay in Gainesville bearable. I would like to thank my friend Reha Uzsoy for being a major influence in my academic orientation and convincing me that "I could do it," and my comrade Meltem Denizel Sivri for always having the time for endless discussions. Last, thanks go to my family for their constant love and support. TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................... iv ABSTRACT ................... ... .....................vii CHAPTERS 1. INTRODUCTION .................... ................ 1 1.1. The Multiple Criteria Decision Making (MCDM) Problem .... .. 2 1.2. Solution Procedures for the MCDM Problem .............. 4 1.3. Vector Maximization Approaches to Problem (MMP) ........ 8 1.4. Optimizing a Linear Function Over the Efficient Set .......... 12 1.5. Contents of the Dissertation ........ .............. .. 13 2. LITERATURE SURVEY ............................... 17 2.1. Methods That Use Prior Articulation of the DM's Preferences 17 2.2. Methods That Use Progressive Articulation of the DM's Preferences (Interactive Algorithms) ......................... 20 2.3. Methods That Use Posterior Articulation of the DM's Preferences .23 2.3.1. Some Conceptual Studies in Vector Maximization ...... 25 2.3.2. Vector Maximization Algorithms for Solving Problem (MMP) ................. ............. 28 2.3.2.1. Nonlinear Vector Maximization Algorithms 28 2.3.2.2. Linear Vector Maximization Algorithms ...... 29 2.3.3. Optimizing a Linear Function Over the Efficient Set of Problem (MOLP) .......................... 36 2.3.4. An Overview of Methods for Finding A Discrete Representation of the Efficient Set ....... ...... 40 2.4. Theoretical Background and Notation ................... .44 3. OPTIMIZATION OVER THE EFFICIENT SET: TOWARDS AN EXACT ALGORITHM ................. .... .... ......... 48 3.1. The BisectionExtreme Point Search Algorithm ............ 49 3.2. An Overview of Concave Minimization Algorithms ......... 52 3.3. Theoretical Background .......................... 54 3.4. The Concave Minimization Algorithm .................. 61 3.5. Validity and Finiteness ......................... .. 66 3.6. Computational Benefits and Preliminary Computational Results 70 4. OPTIMIZING OVER THE EFFICIENT SET: FOUR SPECIAL CASES AND A FACE SEARCH HEURISTIC ALGORITHM ............ 76 4.1. Prelim inaries .................................... 76 4.2. Theoretical Background ........................... 80 4.3. Detecting and Solving the Four Special Cases ............ 99 4.4. The Face Search Heuristic Algorithm ................. 103 4.4.1. General Description ........................ 104 4.4.2. The Face Search Heuristic Algorithm ............. 105 4.4.3. Generating the Sample Sets ........ ...... .... 107 4.4.4. Justification ............................109 4.4.5. Some Implementation Issues ....... ....... .... 110 4.5. Discussion ....... ....... ............ ......... 110 5. FINDING A DISCRETE REPRESENTATION OF THE EFFICIENT SET 113 5.1. Preliminaries ......... .... ...... .. ........... 114 5.2. Theoretical Background ........................... 114 5.3. The Shooting Algorithm ............................ 132 5.3.1. The Algorithm Statement .................... .133 5.3.2. Generating SS .................. .......... 134 5.3.3. Analysis of the Method of Successive Bisection ....... 138 5.4. Examples .................. ................. 140 5.4. Discussion .... ...... ...... .................... 157 6. CONCLUSION AND TOPICS FOR FURTHER RESEARCH ......... 162 APPENDIX CONCAVE MINIMIZATION TEST PROBLEMS, SOLUTIONS AND COMPUTATIONAL RESULTS ............. 171 REFERENCES .... ...... ......... ........................ 182 BIOGRAPHICAL SKETCH ........ ........ ................ 191 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy FINDING DISCRETE REPRESENTATIONS OF THE EFFICIENT SET IN MULTIPLE OBJECTIVE MATHEMATICAL PROGRAMMING: THEORY AND METHODS By SERPIL SAYIN December, 1992 Chairman: Harold P. Benson Major Department: Decision and Information Sciences The main purpose of this dissertation is to formulate methods for representing the efficient set of a multiple objective mathematical programming problem by a discrete sample of points. Finding adequate representations of efficient sets is an important problem in multiple objective decision making. A good representation of an efficient set should consist of a welldispersed set of points that portrays the entire set. We present various algorithms and algorithmic ideas that would contribute to finding such representations with a reasonable amount of computational effort. Solving the problem of optimizing a linear function over the efficient set of a multiple objective linear program can contribute towards the construction of discrete representations of efficient sets. While certain exact and heuristic procedures for solving this problem have been proposed, improved methods are needed. Towards this end, we propose a finite algorithm for concave minimization over a polyhedron which can be used as a subroutine in a recentlyproposed exact algorithm for optimizing over the efficient set. The concave minimization algorithm uses branch and bound and a new idea that we call "neighbor generation." We also study some special cases of optimizing a linear function over an efficient set, and we provide detection and solution procedures for these special cases. To solve the problem, we develop a face search heuristic algorithm which samples among the efficient faces of the feasible region and optimizes the objective function over each efficient face that is found. To develop a direct approach for finding a discrete representation of the efficient set, we introduce the "shooting" idea. This is a global approach and is applicable to the multiple objective mathematical programming problem when the feasible region is convex and the objective functions are linear. After developing the necessary background for the shooting idea, we propose an algorithm for constructing a discrete representation of the efficient set which uses this idea. An important component of the shooting idea is generating discrete representations of a simplex. We present two methods for accomplishing this. One of the methods is probabilitistic, and the other is deterministic. CHAPTER 1 INTRODUCTION Multiobjective mathematical programming has received wide interest in the last twenty years, mostly due to the increasing complexity of real world problems. Many real world problems in which a single alternative is chosen from a number of potential alternatives actually include more than one objective. However, a considerable majority of standard mathematical programming models consist of problems formulated with a single objective function. When the real world problem is indeed a multiple criteria decision making (MCDM) problem, some of these singleobjective models are artificially adapted to the situation by designating one of the objectives as the objective function of the model. The other objectives are built into the constraint set with prespecified levels of achievement and/or limitations (see Anderson 1972 for several examples of such models). These levels, however, are usually not available and difficult to determine. Hence, being able to consider more than one objective simultaneously in a model is of significant value in the sense that it provides a better representation of the decision problem. Throughout this discussion, we will refer to the decision maker (DM) as an individual who faces the problem of choosing one of a number of alternatives. The analyst is the person whose task is to model the DM's problem. 2 The set X c R" of alternatives is the set of programs or plans from which the DM chooses one. The set Y c RP is the set of possible criteria (or attribute) values of the alternatives, where p > 1 is an integer. The criteria function f(x) = [f,(x), f2(x),...,fp(x)], where for each i= 1,..,p, f,: R"R, assigns each alternative x in X a unique set of p attribute (criterion) values. The MCDM problem consists of the triple (X,Y,f). Throughout this dissertation, we will limit our discussion to the case where X is a convex set, and f1, i= 1,..p, are concave functions on X. MCDM was reported to be the most rapidly growing field of OR/MS during the 1970's (Zeleny 1982). The research in this area has its roots in earlier times, however. Kuhn and Tucker (1951) formulated the vector maximization problem and Koopmans (1951) gave proofs of necessary and sufficient conditions for efficiency. Increasing interest in MCDM is primarily due to increasing appreciation of the inherent multiple objective nature of most decision problems, and to rapid improvements in computer technology. The latter development is especially important, since algorithms for multiobjective mathematical programs usually require a significant amount of computation time and storage. 1.1. The Multiple Criteria Decision Making (MCDM) Problem In MCDM, the problem of primary interest is choosing an alternative x = (x,,x2,...,x) in X, i.e. determining the values of decision variables x,, x2,...,x, in order to optimize p objective functions. Henceforth in this dissertation, we will assume 3 that p > 2 for the purpose of eliminating the special case of a single objective function, for which various optimization techniques exist. Without loss of generality, we will assume that the DM wants to maximize each of these objective functions simultaneously. (It is well known that an objective function to be minimized can be modified to fit into this scheme by maximizing the negative of the original objective function.) We can state the multiobjective mathematical programming problem as: VMAX f(x) = [ f,(x), f,(x),..., f(x)] , (MMP) s.t. x E X. Definition 1.1.I. A solution xs to problem (MMP) which maximizes each of the objective functions simultaneously is called a superior solution, i.e., x' is said to be a superior solution if and only if x's X and f,(xs) > f,(x) for i = 1,2,...,p and for all xE X. Definition 1.1.2. The ideal solution for problem (MMP) is a point f' = (f=,f,...,f) in the outcome space such that ft for i = 1,2,..p is the optimal objective value for the problem: Maximize f (x) s.t. x E X. Observe that a superior solution yields the ideal solution for problem (MMP). A superior solution resembles an optimal solution for a single objective optimization problem in the sense that it is a most desirable solution. However, with rare exceptions, the objective functions of problem (MMP) are conflicting in nature, so that no superior 4 solutions exist. In this case, the DM is interested in efficient solutions, also called Paretooptimal or nondominated solutions. Definition 1.1.3. A solution xo E X is an efficient solution of problem (MMP) when x E X, and whenever f(x) > f(x), then f(x) = f(x). Let XE denote the set of efficient solutions for problem (MMP). An alternative x belongs to XE if there exists no other alternative which does at least as well in each objective function and better in at least one objective function. So, under the "more is better" assumption, only the alternatives in the set of efficient solutions of problem (MMP) need to be considered by the DM. In most cases, there will be more than one efficient solution. Therefore, it is often necessary to rank the efficient solutions, or to identify a most preferred solution. To accomplish this, information about the DM's preference structure is required. The time at which this information is needed constitutes a factor of discrimination among various procedures proposed to solve the MCDM problem. 1.2. Solution Procedures for the MCDM Problem In this dissertation, we will partially employ the framework used by Evans (1984). We will review methods classified into three categories on the basis of the time at which the DM needs to articulate his/her preference structure on the set of feasible alternatives X. Although the methods in these categories differ significantly, the goal is to either find a most preferred solution or provide the DM with sufficient information so that he/she can identify a most preferred solution. 5 The first category consists of techniques which require prior elicitation of the DM's preference structure. The methods in this category either seek an explicit formulation of the DM's preferences, as in multiattribute utility theory (Keeney and Raiffa 1976), or implicitly presume a form of preference structure, as in goal programming (Lee 1973), compromise programming (Zeleny 1973), and lexicographic ordering (Keeney and Raiffa 1976). In the multiattribute utility theory approach, the DM's preference structure must satisfy a set of global assumptions, which are often quite restrictive. Through a set of carefully structured questions, the analyst has to determine the DM's value function. The MCDM problem is then usually formulated and solved as a single objective optimization problem yielding a most preferred solution. However, the preference information requested from the DM in these procedures is often difficult to assess. The DM is asked to make value judgements without having enough knowledge of XE, and the responses provided by the DM are required to be more precise than he/she is usually capable of. Methods that do not require the determination of the value function (e.g. goal programming, compromise programming) still require the DM to assess the values of certain parameters. However, since the DM's knowledge about XE is limited, and the effect of the parameters on the solution obtained is not obvious, it is not a simple task to determine these parameters. The methods that use prior articulation of the DM's preferences identify only one solution, which is not usually guaranteed to be an efficient solution. This is not a desirable situation if the "more is better" assumption holds. 6 At the other end of the spectrum are the methods that use posterior articulation of the DM's preferences. These methods attempt to find all or some of the efficient solutions of problem (MMP). The DM then chooses his/her most preferred solution among the solutions thus found. However, characterizing efficient solutions may not be easy unless problem (MMP) possesses a special structure. Therefore, the methods that use posterior articulation of the DM's preferences usually consider special instances of problem (MMP). These approaches may still require intense computational effort. Since these methods convey the maximum information to the DM, the DM may be overwhelmed by the number of efficient solutions being presented. (A more detailed discussion of methods that use posterior articulation of the DM's preferences is deferred to Section 1.3.) Between these two categories are the methods that use progressive articulation of the DM's preferences, generally referred to as interactive algorithms. Interactive algorithms are designed with the purpose of overcoming the difficulties associated with the methods in the other two categories. The major property of the methods in this category is that preference information is requested from the DM as needed during the implementation of the algorithm. This is achieved by an iterative DM/machine interaction. In a typical interactive algorithm, the procedure is initiated by constructing a single objective optimization problem, which is in some way related to the original multiobjective programming problem. Then a solution is found for the single objective problem. Based on the outcome of this solution, the DM is asked to provide some 7 information regarding his/her preferences over the multiple objectives. This information is then utilized in constructing another single objective problem for the next iteration. The procedure terminates when the DM or the computer program identifies the current solution as a most preferred solution. The interactive nature of these methods enhances their practicality. Furthermore, the DM may acquire more information about the efficient set throughout the implementation of the procedure. However, some disadvantages can be stated as follows. a) Interactive methods presuppose an arbitrary degree of precision in the DM's responses. b) The DM is not allowed to decline providing specific information, and cannot alter previous responses. c) Interactive methods require the DM to take an active part each time an optimization problem is solved. d) Some rely on arbitrary rules rather than testable procedures. e) Only one solution is generated although many solutions may be consistent with the DM's preferences. Methods that use prior articulation of the DM's preferences have been the least preferred among the three categories, mainly due to the restrictive assumptions that they require. Methods that use posterior and progressive articulation of the DM's preferences have been the focus of the studies towards solving problem (MMP). The main goal of 8 posterior and progressive methods is to allow the DM to acquire sufficient information about XE so that he/she can identify a most preferred solution. The methods that use posterior articulation of the DM's preferences are also referred to as vector maximization methods. When the objective function and/or the feasible region of problem (MMP) fail to possess some special structure that makes it easy to solve, solving the problem becomes a formidable task even with a single objective function. Therefore, the vector maximization algorithms concentrate on special instances of problem (MMP). We will now briefly discuss the main properties of the vector maximization methods. 1.3. Vector Maximization Approaches to Problem (MMP) Except for a few studies (see Section 2.3.2.1), much of the attention in the area of vector maximization has been on the linear case. Let X c Rn be a nonempty polyhedron. Assume that ci E Rn, i= 1,2,...,p, are row vectors. Let C denote the p xn matrix whose ith row equals ci7, i= 1,2,...,p. Then the multiple objective linear programming problem (MOLP) may be written (MOLP) VMAX Cx s.t. xEX. Problem (MOLP) represents the special linear instance of problem (MMP). In subsequent chapters, we will use the notation c', j = l,..,n, to denote the jh column of C. A number of vector maximization algorithms have been proposed for solving problem (MOLP). A survey of these methods is presented in Chapter 2. It is well known that the set of efficient solutions XE is a union of efficient faces of X. The vector maximization methods usually attempt to identify all of the efficient extreme point 9 solutions and efficient extreme directions of problem (MOLP). The set of efficient solutions presented to the DM then is either the set of all the efficient extreme point solutions thus found, or the set of all efficient solutions characterized as a union of efficient faces. Typically, the suggested algorithms are simplexbased and utilize pivoting operations as a major tool. Hence the computational time required by these algorithms increases with the problem size (see, for instance, Evans and Steuer 1973a, Ecker and Kouada 1978, Isermann 1977). Another complication is the presence of degeneracy, in which case the procedures either lose applicability or become even more cumbersome. Some general difficulties associated with the vector maximization algorithms can be stated as follows. a) When X is a continuous set, XE may be uncountably infinite. It is thus not easy for the DM to choose a most preferred solution from XE. b) The partial solution set obtained by some of the linear vector maximization algorithms, i.e., the set of efficient extreme points, may still be too large, and hence may overwhelm the DM. c) Organizing the efficient extreme points in a presentable manner is a difficult task. d) The set of all efficient extreme points of problem (MOLP) may not provide a good representation of XE. (Steuer [1986] notes that some of the extreme points of a polyhedron may be clustered together, while others may be quite distant from one another.) 10 e) There may not be a most preferred solution for problem (MOLP) that is an extreme point of X. There are also some advantages of the vector maximization approach. Among the methods in the three categories, the vector maximization methods provide the maximum information about XE, and thus have the potential for helping the DM learn about XE in a satisfactory manner. Furthermore, the vector maximization algorithms do not require any preference information from the DM. One way of overcoming the difficulties associated with the vector maximization approach is to obtain a discrete representation of the efficient set. Ideally, this representation should be designed to provide sufficient information about XE without overwhelming the DM. As we have seen, the set of efficient extreme points of X fails to be an ideal representation. These points may not portray XE well, may still be overwhelming, and are usually obtained via computationallyexpensive procedures. To date, no precise method has been proposed to obtain a discrete sample of efficient points that does not consist of solely extreme points, although the idea has occasionally been stated with varying degrees of rigor (Armann 1989, Steuer 1986). However, some suggestions have been made in order to eliminate the problems associated with describing the efficient set of problem (MOLP). The suggested procedures attempt to reduce the number of efficient extreme points presented to the DM. The point estimate weightedsums approach (Steuer 1986) is designed to generate one efficient extreme point of problem (MOLP) at a time. To find each such point, it assigns to each objective function a positive weight and solves the resulting single 11 objective linear program. However, determining the appropriate weighting factors, and varying these weights so as to obtain a representative sample of XE poses a problem that is not easy to solve. Also, although possibly fewer in number, the efficient points obtained as a result of this procedure belong to Xex, which, as we have seen, may not represent XE appropriately. The method of interval criterion weights (Steuer 1976a) designates intervals for the objective function weights rather than point estimates. This method also yields a subset of the efficient extreme points of problem (MOLP) when solved as a vector maximization problem after some restructuring. Although easier than specifying point estimates of weights, there is no a priori method of determining the intervals that would yield a solution set that contains the DM's most preferred solution. Moreover, the resulting vector maximization problem may still require a considerable amount of computation. To help with the problem of organizing the efficient extreme points into a presentable form, the idea of clustering these points has been suggested (Morse 1980). Another suggestion is to eliminate some of the "redundant" efficient extreme points via various techniques of filtering (Steuer 1976b, Steuer and Harris 1980). Clustering and filtering techniques do not save the computational effort required to generate these points. Furthermore, the problem of underrepresenting XE by efficient extreme points is not solved with these approaches. From the above discussion, one can deduce that a good representation of the efficient set should be complete, but should not contain points that are clustered together, 12 and thus are redundant. An approach that may be of practical use in attempting to obtain such a representation is solving the problem of optimizing a linear function over the efficient set. 1.4. Optimizing a Linear Function Over the Efficient Set Recently, the problem of optimizing a linear function over the efficient set of problem (MOLP) has been the focus of several studies. This problem arose at least partially in response to the difficulties in using multiple objective linear programming as a decision aid. If a linear function is available which acts as a criterion for measuring the importance of the efficient solutions of problem (MOLP), then optimizing this function over the efficient set avoids generation of all of the efficient solutions. The problem of optimizing a linear function over the efficient set XE of problem (MOLP) was first introduced by Philip (1972). This problem, denoted as problem (P), is given by (P) max < d,x >, s.t. x E XE, where d E R". Let vd denote the optimal objective function value of problem (P). A special case of problem (P) occurs when d = wTC for some w E RP. In this case, d is linearly dependent on the rows of C, the matrix of objective function coefficients of problem (MOLP). Henceforth, we will denote this special case of problem (P) as problem (PD). An important special case of problem (PD) occurs when d = c, for some i E {1,2,...,p}. In this case, problem (PD) seeks the minimum value of the objective function 13 can be specialized to solve the problem of minimizing an objective function of problem (MOLP) over X, serves several useful purposes of its own in multiple criteria decision making. The optimal value of this problem, along with the maximum value of the same objective function over XE, defines the range of values that over XE Knowledge of this range aids the decision maker in setting goals, evaluating points in XE and ranking objective functions (Isermann and Steuer 1987). In a way, the points that achieve the maximum and minimum of objective functions constitute a discrete representation of the efficient set, albeit very sparse. This representation, however, helps in defining the extremities of the efficient set in terms of objective function values. Therefore, along with other ideas, this particular approach may be of relevance in finding a discrete representation of XE. A detailed discussion of the solution procedures proposed for problem (P) is deferred to Chapter 2. 1.5. Contents of the Dissertation Among the three categories of approaches, most of the recent work towards solving problem (MMP) has concentrated on approaches that use progressive or posterior articulation of the DM's preferences. Methods that use prior articulation of the DM's preferences generally suffer from either demanding too much preference information from the DM, or making restrictive assumptions in order to represent the problem. However, along with the advantages that make progressive and posterior approaches more favorable, these approaches also have various disadvantages (cf. Section 1.2 and 14 Section 1.3). Thus, none of the three types of approaches is fully adequate as a means for solving problem (MMP). Interactive algorithms are usually designed to enhance computational practicality. This is usually achieved at the expense of relying on arbitrary rules. They may require preference information from the DM that is not easy to assess, and may not provide sufficient information about XE. Vector maximization algorithms do not require any preference information from the DM. They are usually unwieldy in terms of computational requirements. They are designed to provide the maximum information about XE. This information may consist of either all of the efficient solutions, or all of the efficient extreme point solutions. However, the information generated as a result of these procedures may be difficult to organize and present, and is usually overwhelming to the DM. Furthermore, if the suggested solution consists of the extreme points that belong to XE, this discrete representation may fail to portray XE in a comprehensive way. In the dissertation, we will concentrate on the problem of finding a discrete representation of the efficient set of problem (MMP). Our ultimate goal is to find a "welldispersed," but adequate discrete representation of the efficient set with minimal computational effort. We will present a procedure for obtaining a discrete representation that does not consist of solely extreme points. The procedure will not require any preference information from the DM prior to, or during implementation. No attempts will be made to identify all of the extreme points that belong to XE. 15 In this way, finding a discrete representation of XE combines the advantages of progressive and posterior approaches while eliminating their most common disadvantages. We also study the problem of optimizing a linear function over the efficient set. The fact that problem (P) can be specialized to find the criterion ranges over the efficient set helps in representing the efficient set. Moreover, under certain assumptions, solving problem (P) eliminates the need for the representation of all of XE. A literature survey of the MCDM problem will be presented in Chapter 2. Emphasis will be placed on the vector maximization approach while methods that use prior and progressive articulation of the DM's preferences will be briefly reviewed. Algorithms for solving the problem of optimizing a linear function over the efficient set will also be discussed. Chapter 3 contains some studies towards solving problem (PD) optimally. Our work is motivated by a recent algorithm for solving problem (PD) (Benson 1991c). The algorithm calls for repeatedly solving certain similar concave minimization problems in one of its steps. Since the major computational requirement of Benson's algorithm is solving these problems, it is important that an efficient procedure be employed. We will present a finite branch and bound algorithm with a neighbor generation process to minimize a concave function over a compact polyhedron. In Chapter 4, we study the problem of optimizing a linear function over the efficient set. Some results pertaining to certain special cases of the problem of optimizing a linear function over the efficient set of problem (MOLP) will be presented. Methods for solving problem (P) optimally when any of these special cases is 16 encountered will be given. A heuristic algorithm for solving problem (P) will be proposed. Our ideas for finding a discrete representation of the efficient set of problem (MMP) will be presented in Chapter 5. In the special instance of problem (MMP) that we consider, the objective functions of problem (MMP) are assumed to be linear, and the feasible region is assumed to be compact and convex. We will introduce a global approach to the problem of obtaining a representative sample of efficient points. We will give an algorithm that is designed to obtain a good representation of XE that is well dispersed, and that does not consist solely of extreme points. Finally, in Chapter 6, an overall summary and conclusions will be given, and directions for further research for each problem will be discussed. CHAPTER 2 LITERATURE SURVEY In this chapter, we will present a literature survey of the methods that have been proposed to solve problem (MMP). Since our research is in the area of vector maximization, emphasis will be placed on the methods that use posterior articulation of the DM's preferences. In Sections 2.1 and 2.2 we will discuss the methods that use prior and progressive articulation of the DM's preferences, respectively. We will present some samples of methods in these two categories rather than a complete review. Our goal is to give the reader a flavor of the methods that use prior and progressive articulation of the DM's preferences. In Section 2.3, a more comprehensive discussion of the vector maximization approach will follow. Also, in Section 2.3, we will present reviews of the methods and ideas that have been suggested for solving the problem of optimizing a linear function over the efficient set, and for finding a discrete representation of the efficient set. 2.1. Methods That Use Prior Articulation of the DM's Preferences Methods that use prior articulation of the DM's preferences can be characterized by one general aspect they share. These methods are all based on the idea of the DM's value function. 18 Definition 2.1.1. A function v, which associates a real number v(f(x)) to each x E X is said to be a value function representing a particular DM's preference structure provided that for each x', x2 E X, 1) f(x') f(x2) if and only if v(f(x')) = v(f(x2)), and 2) f(x') > f(x2) if and only if v(f(x')) > v(f(x2)), where f(x') f(x2) denotes that the DM is indifferent between outcomes f(x') and f(x2), and f(x') > f(x2) denotes that the DM prefers outcome f(x') to outcome f(x2). A more detailed discussion of value functions, which are sometimes referred to as utility functions, can be found in Keeney and Raiffa (1976). Methods that use prior articulation of the DM's preferences differ in the way they utilize the value function. There have been two approaches in MCDM using the concept of a value function. One approach is to explicitly use a value function. This approach is known as multiattribute utility theory (Keeney and Raiffa 1976). This technique involves two steps: i) find the value function v of the DM, ii) solve the single objective mathematical programming problem (U) Max v[ f1(x), f2(x),..., fp(x) ], s.t. xEX, for an optimal solution x', which is called a best compromise solution or a most preferred solution. 19 As mentioned earlier, the multiattribute utility theory approach makes explicit use of the value function. This, however, constitutes a disadvantage for the implementation of the technique. In order to find the value function, the DM's preferences among the alternatives in X must satisfy a set of global assumptions. To apply the technique, the analyst, by asking the DM a carefullystructured set of questions, first has to verify that the global assumptions are satisfied. Although alternative forms of the global assumptions may be stated, their verification is not an easy task. Furthermore, if the assumptions fail, the analyst cannot continue because no value function is obtained. If the global assumptions are satisfied, then the value function can be obtained by asking the DM more questions. The value function aggregates the p criteria into one function. This enables the analyst to solve the single objective optimization problem (U) to obtain the DM's most preferred solution. When the components of problem (U) have certain characteristics, solving problem (U) becomes relatively easy. If the assessment of the value function is completed successfully, and a solution is found, although it is guaranteed to be a most preferred solution, it is not guaranteed to be efficient unless the value function of the DM is strictly increasing in its arguments. A second approach using a priori articulation of the DM's preferences is to implicitly use a value function, as in goal programming (Lee 1973), compromise programming (Zeleny 1973), and lexicographic ordering (Keeney and Raiffa 1976). Since the derivation of the value function is quite cumbersome, dealing with this issue implicitly yields a more practical approach. Although much easier to implement than the 20 multiattribute utility theory approach, the implicit assumptions associated with the methods using this approach may be restrictive. Furthermore, the methods yield only one solution, whereas a number of solutions might be consistent with the DM's preferences. They also fail to guarantee that the solution found is efficient. For further details on the methods that use a priori articulation of the DM's preferences, the reader is referred to Kornbluth (1973), Huber (1974), Keeney and Raiffa (1976), Farquhar (1977). 2.2. Methods That Use Progressive Articulation of the DM's Preferences (Interactive Algorithms) Interactive algorithms have the objective of eliminating the difficulties that arise when the methods that use prior articulation of the DM's preferences are employed. As in the methods that belong to the first category, the emphasis is on finding a single solution that the DM most prefers. However, the DM's preferences are used while the technique is being used rather than before the implementation of the method. Algorithms in this category often involve an interactive DM/computer approach. During the initial step of a typical interactive algorithm, an initial feasible solution is obtained, often via a single objective optimization problem which is in some way related to the original multiobjective problem. Next, the DM is asked to provide some information about his/her preferences with regard to the outcome of this solution. Based on the answer of the DM, the computer provides another solution, again probably via a single objective optimization problem. The procedure continues in this manner until the DM decides that the current solution is a most preferred solution to the MCDM problem. 21 The algorithms in this category differ in the type of the single objective optimization problem solved at each iteration and/or in the information elicited from the DM (Kok 1986, Ringuest and Gulledge 1985). Although some of these algorithms involve elicitation of some tradeoff information, which may be difficult to assess (Dyer 1973, Wallenius 1975), in general, the required information is easier for the DM to provide compared to the methods in the first category. One of the earliest algorithms in this category is that of Geoffrion, Dyer and Feinberg (1972). Their procedure is based on the assumption that the DM's value function indeed exists, and is concave and nondecreasing in its arguments, but is not available. Their algorithm applies to problem (MMP) when X is a compact polyhedron, and fi, i= 1,..,p are concave and differentiable. Therefore, if the DM's value function were available, problem (U) would be a simple convex program, and could be solved to obtain a most preferred solution. The key idea of the method is to implement the Frank Wolfe Method on problem (U) without explicitly knowing the value function v, and to ask the DM appropriate questions when some information about v is needed. Marcotte and Soland (1986) presented a particularly inventive interactive branch and bound algorithm for solving problem (MMP). They suggested solving an equivalent transformed problem in the objective function space. Their algorithm is significant in more than one respect. First, it does not require even the assumption of the existence of a value function. Rather, it is assumed that the DM's preference structure ensures the existence of indifference curves. Second, it can be applied to problem (MMP) when Y 22 is either convex or discrete. Finally, it provides an efficient solution even if the DM decides to stop before the algorithm terminates. In general, interactive algorithms tend to be much more practical than the algorithms that require prior articulation of the DM's preferences. Specifically, information demanded from the DM by the interactive methods is more reasonable than in the methods in the first category. The interactive methods are also usually computationally tractable. Moreover, interactive algorithms let the DM see various feasible solutions, explore the feasible region somewhat, and learn some of the tradeoffs available in the objective functions. This enables the DM to understand the chosen solution better and to have more confidence in it. An ultimate superiority in terms of practicality of the interactive algorithms over the methods that utilize prior articulation of preferences is that there is no explicit or implicit attempt to find the DM's value function. However, there are also a number of disadvantages associated with the methods that use progressive articulation of the DM's preferences. Many require more precision in the DM's responses than is possible, and usually, previous responses cannot be altered. Only one solution is generated, even though a whole set of solutions may be consistent with the DM's preferences. Furthermore, in many interactive algorithms, the solutions generated throughout the application of the algorithm, including the solution suggested as the most preferred solution, may not be efficient. Typically, only a few points in X or XE are explored while implementing an interactive algorithm. Thus the DM may not be provided with sufficient information about XE. Potentially, the solution 23 that is identified as the most preferred may be less preferred to some other efficient solution that lies in some unexplored portion of X. Finally, most of the algorithms in this category suffer from being ad hoc in the sense that they rely on arbitrary rules rather than testable, logical rules related to the value function. Recently, interactive algorithms for solving problem (MMP) have been very popular. This is mostly due to the progress in microcomputer software. An enormous number of interactive algorithms have been proposed (see Larichev 1971, Wallenius 1975, Steuer 1977, Hwang and Masud 1979, White 1980, Evans 1984, Hazen 1980). A recent bibliography of interactive algorithms is given by Aksoy (1989). 2.3. Methods That Use Posterior Articulation of the DM's Preferences The methods in this category do not utilize the DM's preferences until after the method has been employed. Unlike the other two categories, these methods try to find many solutions rather than just one. The general procedure consists of two stages. First, the method is used to generate all or many efficient solutions to problem (MMP). Next, the set of solutions generated is presented to the DM, and he/she chooses his/her most preferred solution. As implied by the statement of the general procedure, no explicit preference information is asked directly of the DM. Instead, only a single implicit preference assumption is made. This assumption can be expressed as "more is better," i.e. it is assumed that the DM would prefer attaining a higher level in any of the objective functions to attaining a lower level of the same objective function. This constitutes one 24 of the major advantages of the methods that use posterior articulation of the DM's preferences. However, the task of identifying all of the efficient solutions is a formidable one. Moreover, the number of solutions being presented to the DM may be overwhelmingly enormous. To represent approaches that use posterior articulation of the DM's preferences, the term vector maximization has often been used. Vector maximization was originally defined as the problem of finding all of the efficient solutions of problem (MMP). The algorithms that have been proposed to solve problem (MMP) mostly concentrate on one special instance of the problem, namely problem (MOLP). While some of these algorithms suggest finding all of the efficient solutions of problem (MOLP), there has been a trend towards concentrating on finding a subset of the efficient set of problem (MOLP) which, in some sense, represents the efficient set. Most of the work towards more general cases of problem (MMP) has been conceptual. Apart from algorithmic approaches that have concentrated on problem (MOLP), solution procedures for a few other special cases of problem (MMP) have been suggested. In Section 2.3.1, we will present a review of some of the conceptual and theoretical work in the area of vector maximization. In Section 2.3.2, we will review some of the vector maximization methods that have been proposed to solve problem (MMP) with special emphasis on problem (MOLP). Section 2.3.3 will contain an overview of the methods that have been proposed to solve the problem of optimizing a linear function over the efficient set. In Section 2.3.4, we will review the methods that 25 have been suggested for finding discrete subsets of representative points in the efficient set of problem (MOLP). 2.3.1. Some Conceptual Studies in Vector Maximization In one of the earliest studies in the field of vector maximization, Kuhn and Tucker (1951) defined the concept of proper efficiency, which eliminates certain efficient points that exhibit an undesirable anomaly. Later, Geoffrion (1968) noted that efficient solutions may exist which demonstrate a similar anomaly, but are proper in the sense of Kuhn and Tucker. He thus redefined proper efficiency. An efficient solution that is not properly efficient is said to be improperly efficient. If xo is an improperly efficient solution, for some criterion i, the marginal gain in f, can be made arbitrarily large relative to each of the marginal losses in other criteria. Since the DM's desire for fi is not satisfied, xo is not a desirable solution. Also, for nonlinear problems, the concept of proper efficiency allows for a more appropriate characterization than does the concept of efficiency. Hence it seems reasonable to consider only the properly efficient solutions for problem (MMP). Geoffrion provided important results that may be functional in obtaining properly efficient solutions, particularly under the assumption of concave objective functions. He also defined six different problems related to proper efficient solutions and showed their connections under certain assumptions. Later, Benson and Morin (1977) gave necessary and sufficient conditions for an efficient solution for problem (MMP) to be properly efficient. These conditions relate the proper efficiency of a given solution to the stability of certain single objective 26 maximization problems. A direct consequence of this theory is that any efficient solution for problem (MOLP) is properly efficient. (Also see Benson 1983, Isermann 1974). The concept of proper efficiency is important in the sense that it enables certain characterizations that may have computational implementations. Furthermore, the exclusion of improperly efficient solutions in problem (MMP) does not constitute a critical problem, since under certain assumptions (including the assumptions of a convex feasible region and concave objective functions), the outcome of any improper solution of (MMP) is the limit point of the outcomes of some sequence of properly efficient solutions (Benson 1978). Yu (1974) proposed a domination structure over the objective function space and explored the geometry of the set of all nondominated solutions. The definition of a nondominated solution is a generalization of that of an efficient solution. He also introduced the concepts of cone convexity and cone extreme point, and investigated their main properties. Cone convexity can be regarded as a generalization of the concept of a convex set. Yu derived sufficient conditions for Y to be cone convex through the concepts of polar cones and polyhedral cones. He showed that a nondominated solution must be a cone extreme point. Based on this result, he outlined a sequential approximation idea for locating the set of all nondominated solutions. In his study, Yu (1974) introduced two approaches for locating the set of all nondominated solutions in the decision space through ordinary mathematical programming problems. Two related approaches to locate all nondominated solutions in 27 the objective function space were also presented. Efficient solutions (see Definition 1.2.3) were studied as a special case of nondominated solutions. In his pioneering study, Philip (1972) introduced algorithmic frameworks for four different problems that are of importance in the area of linear vector maximization. Although his methods are referred to as "algorithms," we find it more appropriate to review his work in this section, since they are closer to being ideas than readily implementable algorithms. Throughout his study, Philip considered linear objective functions and a convex (in certain cases polyhedral) feasible region. He also presented necessary and sufficient conditions for a point on the boundary of the feasible region X to be an efficient solution for problem (MMP) when X is a polyhedral set. In general, Philip's methods use linear programming as a tool to solve the stated problems. The problems introduced by Philip have been studied by many researchers. The first problem he considered is the problem of testing for efficiency of a given point. He provided two methods for the cases where X is polyhedral and where the number of constraints that define X and the number of objective functions are allowed to be infinite. The second problem is the problem of finding an initial efficient point in X for use in interactive or vector maximization algorithms. The third problem that he studied is the problem of testing to see if a given efficient point is the only efficient point that exists, and if not, finding other ones. The last problem, which has been the focus of several very recent studies, is the problem of optimizing a linear function over the set of efficient solutions. We will discuss this problem in more detail later. 28 A special case of problem (MMP) occurs when fi = xi for each i= l,..n. In this case, the efficient points of problem (MMP) are called admissible points. The motivation for studying admissible points of problem (MMP) comes from various sources. First, since this constitutes a special case of problem (MMP), some results that are not valid for the general case can be developed. Second, when instead of problem (MMP), an equivalent transformed problem in the objective function space is considered, efficient solutions of problem (MMP) map into the admissible points of the transformed problem. Some properties of admissible points have been studied by a few authors. The interested reader is referred to Arrow, Barankin and Blackwell (1953), Bitran and Magnanti (1979), and Benson (1982). 2.3.2. Vector Maximization Algorithms for Solving Problem (MMP) Although most of the vector maximization algorithms that have been proposed for solving problem (MMP) concentrate on problem (MOLP), a few algorithms have been suggested for some other special cases where X is nonpolyhedral or fi, i=l,..,p, are nonlinear. We will first study the nonlinear vector maximization algorithms. 2.3.2.1. Nonlinear vector maximization algorithms The vector maximization methods that have been proposed to solve nonlinear instances of problem (MMP) concentrate on special cases. This is due to the inherent difficulty of the general problem. Benson (1979) considered the special case of problem (MMP) with p=2 objective functions. He provided a characterization for efficient points for this special case. He 29 then outlined a parametric procedure for generating the set of all efficient points for this special instance of problem (MMP). Another special instance of problem (MMP) is the multiobjective location problem. In this special case, X C R2, and the problem is to locate a new facility, i.e. determine (x,, x2), so as to minimize the "distance" between the new facility and each of the p existing facilities. Therefore the objective associated with each existing facility is to locate the new facility as close as possible to that facility. Wendell, Hurter and Lowe (1977) studied the multiobjective facility location problem when the distance is defined as the rectilinear distance. They developed some properties of XE and gave two procedures for generating XE. Later, Chalmet, Francis and Kolen (1981) studied the same problem as Wendell, Hurter and Lowe, and developed two algorithms for it. Both of their algorithms generate XE. They also established that their second algorithm has a computational complexity that is the best possible among all the algorithms designed for performing this task. Although the methods we have reviewed in this section are valuable, the problem of solving more general instances of problem (MMP) still needs to be studied. 2.3.2.2. Linear vector maximization algorithms It is well known that the efficient set of problem (MOLP) can be represented as a union of the efficient faces of the feasible region. Algorithms that are suggested to solve problem (MOLP) seek to identify either all of the efficient extreme points of X, or all of the efficient faces of X. Although the proposed algorithms differ in certain ways, they all utilize some modified version of the simplex method. Usually, some sorts 30 of tests are incorporated into the solution procedures in order to identify pivots that lead to efficient vertices or directions (Ecker and Kouada 1978, Isermann 1977, Evans and Steuer 1973a). In some procedures, these tests also produce the efficient faces incident to the current vertex (Ecker, Hegner and Kouada 1980, Gal 1977). Depending upon the structure of the problem, and especially the dimension of the decision space, finding all of the efficient vertices may become an overwhelming assignment. Hence, it is not very surprising that these algorithms require a considerable amount of computer time and storage. Worse yet, in the presence of degeneracy, either the procedures become more cumbersome or inapplicable. One of the earliest algorithms proposed for solving problem (MOLP) is that of Evans and Steuer (1973a). In their study, Evans and Steuer presented a corollary which states that if there exists an efficient solution to problem (MOLP), then at least one extreme point efficient solution exists. They also provided a test for the efficiency of an extreme point. This test is a variant of a similar formulation given in Philip (1972) for a similar purpose. Evans and Steuer (1973a) classified multiple objective linear problems into five mutually exclusive (and jointly exhaustive) groups, and they provided results to identify the group to which a particular multiple objective programming problem belongs. They then presented a sequential maximization technique which either yields an efficient extreme point, if one exists, or terminates with the indication that the efficient set is empty. 31 The algorithm of Evans and Steuer is a modification of the simplex method. They also provided results of some computational experiments in which they analyzed and compared four different options for implementing their algorithm. In some of these options, they incorporated subproblem tests into the revised simplex algorithm. These subproblem tests are utilized in testing the transient extreme points generated during the implementation of the simplex method to see if they are efficient or not. Their results show that frequent calls to subproblem tests worsen the computational performance of the procedure. One important conclusion that can be drawn from these experiments is that the average computational time increases as the number of criteria increases. The procedures in Evans and Steuer (1973a), as presented, do not incorporate techniques to handle degeneracy in these problems. Evans and Steuer (1973b) presented two revised simplex, subproblemoriented algorithms. Each algorithm first finds an efficient extreme point, then enumerates all remaining efficient extreme points. The first algorithm is an adjacent efficient extreme point method which involves the examination of the efficiency of all edges emanating from a given extreme point. The second algorithm is an adjacent efficient basis method which involves pivoting among the set of all efficient bases. Although the first algorithm is structured to deal with degeneracy, the second is not. Later, Yu and Zeleny (1975) concentrated on the problem of finding the set of all nondominated solutions associated with problem (MOLP). They showed that the set of all dominated solutions is convex and, when X is compact, that the set of all 32 nondominated solutions is a subset of the convex hull of the nondominated extreme points. They also showed that the set of nondominated extreme points of problem (MOLP) is connected, i.e. given two efficient extreme points, an ordered set of adjacent efficient extreme points exists whose first element is one of the points and whose last element is the other point. Yu and Zeleny next defined the faces of a polyhedron incident to a particular extreme point as a function of the set of nonbasic indices at that extreme point. In addition to a decomposition theorem, they derived some necessary and sufficient conditions for a face to be nondominated. Based on these results, Yu and Zeleny presented two methods. Their first method is a multicriteria simplex method which identifies all nondominated extreme points of problem (MOLP). For simplicity, their method assumes that the feasible region is compact and nondegenerate. Their second method focused on the problem of generating the entire set of nondominated solutions. This method consists of systematically obtaining efficient extreme points and investigating faces of decreasing dimension incident to these points in order to identify efficient ones. Ecker and Kouada (1978) presented an algorithm for finding all of the efficient extreme points of problem (MOLP). They provided a characterization for the efficiency of an edge incident to an efficient extreme point. Their algorithm utilizes this characterization to obtain all of the efficient extreme points of X. 33 Ecker and Kouada (1976) also provided useful results to characterize efficient faces. A face is called maximally efficient if there exists no other higher dimensional efficient face that contains it. They developed an algorithm called FACE to find all maximally efficient faces incident to a given vertex of the feasible region of problem (MOLP) under the assumption of nondegeneracy. In the algorithm FACE, first, efficient edges incident to the given vertex are isolated. Since these are the only edges that can possibly belong to efficient faces containing the vertex, the effort involved in checking all possible faces incident to the given vertex for efficiency is avoided. Algorithm FACE then sequentially constructs efficient faces incident to the vertex, each one a proper subset of its successor, until a maximal efficient face is obtained. Then, other maximal efficient faces are obtained by backtracking. Later, Ecker, Hegner and Kouada (1980) combined algorithm FACE with the algorithm of Ecker and Kouada (1978). In this way, they developed an algorithm for generating all maximal efficient faces for problem (MOLP). In Ecker, Hegner and Kouada (1980), a maximal face is characterized as the optimal solution set for a particular linear program. Isermann (1977) used a dual approach to multiparametric linear programming in order to identify all efficient solutions of problem (MOLP). The procedure he proposed first identifies all efficient extreme points and efficient extreme directions of problem (MOLP). The set of all efficient solutions is then determined as a union of a minimal number of convex sets of efficient solutions. 34 All the methods that we have reviewed so far attempt to identify all efficient extreme point solutions to problem (MOLP). The set presented to the DM is either the set of all efficient extreme points, or the set of all maximally efficient faces characterized in certain ways. Starting from an initial efficient extreme point solution (see for instance, Ecker and Hegner 1978, Benson 1981, Benson 1991a for finding an initial efficient extreme point solution), other efficient extreme points are obtained by certain pivoting operations. Since the set of efficient extreme points of problem (MOLP) is connected, a path is traversed along efficient extreme points, and some bookkeeping procedures are employed in an effort to enhance practicality. To generate efficient faces, special subproblems are used. All of these procedures are organized into an accounting scheme in order to make the techniques implementable. One general difficulty that arises in all of these procedures is degeneracy. It is wellknown that several bases can be assigned to a single degenerate vertex. Since all of these methods use the notion of a basic feasible solution to characterize the vertices of the feasible region, the task of determining the set of all efficient vertices becomes more complicated when degeneracy is present Murty (1985) showed that the standard methods of perturbation or lexicographic order may not suffice to overcome the difficulties that arise in the presence of degeneracy. He then provided a generalization of the representation of faces of a polyhedron in an effort to eliminate the complications caused by degeneracy. His proposed method is a modification of the Ecker, Hegner, Kouada algorithm (1980) and provides a framework for obtaining all of the efficient faces 35 incident to a degenerate extreme point. However, the task still remains a computationallyprohibitive one. As an alternative to the pivoting based approaches, Gal (1977) proposed a procedure that relies on multiparametric linear programming. His method eliminates the need for explicit treatment of degeneracy and unbounded feasible regions. However, solving the associated multiparametric linear programs may have heavy computational requirements. In a more recent study, Armand and Malivert (1990) developed a twostep method for finding the set of efficient solutions of problem (MOLP). They gave special pivoting rules in order to avoid generating all of the bases that correspond to a degenerate efficient vertex. Their method first finds all of the efficient extreme points. Then, by a combinatorial process, faces incident to each of these vertices are constructed, and maximally efficient faces are identified. Later, Armand (1990) presented another method in which a local approach is employed in order to reduce the computational effort required by the combinatorial process in Armand and Malivert (1990). Although this method practically outperforms the previous one, the results display a significant increase in computational time when degeneracy is encountered. To summarize, the vector maximization algorithms for problem (MOLP) generally seek to identify all of the efficient extreme points, or all of the efficient faces of X. In that respect, they are designed to convey the maximum information to the DM. This contrasts with the methods that use prior and progressive articulation of the DM's 36 preferences. However, this is not necessarily an advantage. Apart from the computational burden of employing these methods, and the complications induced by the presence of degeneracy, there are a number of other problems associated with vector maximization algorithms. a) The information being presented to the DM is usually overwhelming. b) When the set of all efficient solutions is obtained, presenting these to the DM in a form that would make it easy to understand is difficult. c) The number of efficient extreme points of X may grow considerably as a function of problem size (Dauer 1990). Therefore, even approaches that only generate the set of efficient extreme points of X can overwhelm the DM. d) The set of all efficient extreme points may not constitute a good representation of the efficient set (cf. Section 1.3). The factors that are listed above have raised the issue of finding a better representation of XE. We will discuss this issue in more detail in Section 2.3.4. 2.3.3. Optimizing a Linear Function Over the Efficient Set of Problem (MOLP) Problem (P), the problem of optimizing a linear function over the efficient set XE of problem (MOLP), was introduced by Philip (1972). Problem (P) arose at least partially in response to some of the difficulties involved in using problem (MOLP) as a decision aid. By solving problem (P) instead of problem (MOLP), the computational burden of generating all of XE or all of the extreme points of XE is avoided (Ecker and Kouada 1978, Isermann 1977, Yu and Zeleny 1975). Moreover, since problem (P) yields 37 only one efficient point, the danger of possibly overwhelming the decision maker with large numbers of efficient points to choose from is precluded. Mathematically, a special case of problem (P) is problem (PD), where d = wTC for some w E RP. In particular, when d = ci for some i E {1,2,...,p}, problem (PD) seeks the minimum value of the objective function < ci,x > over the efficient set of problem (MOLP). The fact that problem (P) can be specialized to solve the problem of minimizing an objective function several useful purposes of its own in multiple criteria decision making. The optimal value of this problem, along with the maximum value of the same objective function over XE, defines the range of values that < c,,x > takes over XE Knowledge of this range aids the decision maker in setting goals, evaluating points in XE and ranking objective functions (Isermann and Steuer 1987). Furthermore, by obtaining the points that achieve the minimum and the maximum values of the objective functions over XE, a discrete representation of XE, albeit very sparse, is obtained. Since XE is generally a nonconvex set, problem (P) can be classified as a global optimization problem. It is well known that such problems possess local maxima which need not be globally maximal. Moreover, the number of local maxima may increase considerably as the number of variables in a global optimization problem increases. Therefore, global optimization problems are much more difficult to solve than convex programming problems. 38 In spite of the importance of problem (P), only a few procedures have been proposed for globally solving it. This might be partially due to the inherent difficulty of finding a global maximum for the problem. The first method to be proposed for optimally solving problem (P) was given by Philip (1972). As part of a larger study of problem (MOLP), Philip briefly outlined a cutting plane procedure for solving problem (P). Later, the same procedure was given by Isermann and Steuer (1987) for the special case of minimizing an objective function of problem (MOLP) over XE. In both algorithms, each time a cutting plane is introduced, it is necessary to search along the intersection of the cutting plane and the current feasible polyhedron for all newlycreated extreme points. Since no mathematical procedure is given to describe how to perform this search, it is not clear how to, or whether it is possible to, implement these algorithms. Isermann and Steuer (1987) proposed another algorithm for minimizing an objective function over the efficient set. Their algorithm requires solving mathematical programs that contain nonconvex quadratic constraints. Benson (1984) studied various aspects of problem (P). He derived necessary and sufficient conditions for problem (P) to be unbounded. He also elaborated certain properties of the optimal solution set of problem (P). Specifically, the results given by Benson concerning polyhedral feasible regions suggest potential computational procedures for finding an optimal solution to the problem. 39 More recently, Benson proposed three algorithms for solving problem (P) (1991b, 1992, 1991c). These algorithms are all finite and can be implemented using only linear programming techniques. The first implementable algorithm for solving problem (P) given by Benson (1991b) actually solves an infinitelyconstrained problem which is equivalent to problem (P). His algorithm uses relaxation to solve this problem. The algorithm stops either with an extreme point or nonextreme point optimal solution for the problem after a finite number of iterations. The second algorithm proposed by Benson (1992) is an extreme point search algorithm. In the initial step of the algorithm, an initial efficient extreme point is found. Then, in each iteration, an improved efficient extreme point is found which is distinct from all previouslyobtained efficient points. This new point need not be adjacent to the point obtained in the previous iteration. The algorithm finds a globally optimal extreme point solution for problem (P) in a finite number of iterations. The third algorithm proposed by Benson (1991c) is designed for problem (PD). The algorithm uses a bisection search to find progressively smaller intervals of the real number line that contain the optimal value of problem (PD). For each value of the parameter chosen during the bisection search, a convex maximization problem over X is solved. Depending upon the value of the objective function of the convex maximization problem, an ordinary local optimum search procedure may be invoked in some iterations of the bisection algorithm. The algorithm terminates with an extreme 40 point optimal solution for problem (PD) in a finite number of steps, provided that a finite algorithm is employed to solve the convex maximization subproblems. Although these algorithms are promising and have various advantages, they suffer from the requirement of solving certain nonconvex optimization problems in each iteration. Heuristic algorithms that are proposed to approximate a global optimal solution for problem (P) are quite sparse. Dessouky, Ghiassi and Davis (1986) presented three heuristic algorithms for the special case of minimizing an objective function over the efficient set. Their procedures suggest initiating one of the search procedures for globally solving problem (P), but terminating it prematurely with a locally optimal solution. One of these procedures is complicated by the requirement of finding good solutions to certain bilinear programming problems. Dauer (1991) proposed a heuristic algorithm for solving problem (P). His algorithm attempts to find a local optimal solution by searching adjacent efficient faces of X using an active constraint approach. 2.3.4. An Overview of Methods for Finding A Discrete Representation of the Efficient Set Among the three main categories of approaches that we have reviewed so far, methods that use prior articulation of the DM's preferences have been the least favored. Methods in this category usually suffer from making restrictive assumptions, or demanding too much information from the DM. Hence, methods that use progressive or posterior articulation of the DM's preferences have been the most studied. However, 41 both types of approaches have certain advantages and disadvantages. Thus neither category of methods dominates the other. In general, interactive methods presuppose an arbitrary degree of accuracy in the DM's responses, usually rely on arbitrary rules, and fail to provide sufficient information to the DM. On the other hand, vector maximization algorithms usually concentrate on solving problem (MOLP), require heavy computations, and have the risk of overwhelming the DM by the abundance of efficient points being generated. Towards this end, obtaining a good discrete representation of the efficient set of problem (MMP) would eliminate the problems associated with both types of approaches. A discrete representation of XE obtained by the vector maximization algorithms reviewed in Section 2.3.2.2 is the set of all efficient extreme points of problem (MOLP). However, aside from the heavy computational burden of generating all the efficient extreme points, this is not a very desirable representation. The efficient extreme points being presented to the DM may not be welldispersed, and thus may not constitute a good representation. It becomes evident from the above discussions that a "good" representation of the efficient set of problem (MOLP) should possess certain properties, such as being well dispersed and having a relativelysmall cardinality. Clearly, it would be preferable to attain this representation with a reasonable amount of computational effort. Thus efficiently finding a good discrete representation of XE would constitute a practical way for providing the DM with sufficient information for choosing a most preferred solution. 42 In this way, an approach of seeking a good discrete representation of XE could combine the best properties of progressive and posterior approaches. The problems associated with the solutions obtained by a vector maximization algorithm have been noticed by a few authors. Some suggestions have been made in order to remedy at least some of the disadvantages of vector maximization algorithms. One way of overcoming the difficulty associated with the vector maximization approach is assigning positive weights to the objective functions of problem (MOLP) and solving the resulting single objective linear program. This can also be regarded as a special case of the multiattribute utility theory idea (Keeney and Raiffa 1976). This method would generate only one efficient extreme point. However, finding an accurate estimate of the weights that would yield the DM's most preferred solution is not easy. Hence this approach does not provide a desirable representation (see, for instance, Eckenrode 1965, Steuer 1986 and references therein). In an attempt to reduce the number of efficient extreme points obtained, Steuer (1976a) suggested assigning interval criterion weights to the objective functions of problem (MOLP). In this method, interval estimates of weights are determined rather than point estimates. Steuer gave a procedure that involves converting the multiple objective linear programming problem with interval criterion weights into an equivalent vector maximization problem. The set of all the efficient extreme points of the converted problem comprises a subset of the set of all efficient extreme points of the original problem. Since this procedure requires finding all the efficient extreme points of the converted problem, it resembles the methods discussed in Section 2.3.2.2. In particular, 43 its computational requirements are heavy, and the solutions generated for problem (MOLP) consists of extreme points. Thus the solutions being presented to the DM may not constitute a good representation of XE. In an attempt to obtain a small sample of efficient points with welldispersed locations, Steuer introduced a filtering technique (Steuer 1976b, Steuer and Harris 1980, Steuer 1986). Prior to the application of this technique, first either the set of all efficient extreme points or a subset of this set is obtained. Then an appropriate distance function is chosen. Filtering is then used to eliminate some of the points in this sample based on the distances between various pairs of points. Filtering fails to be an ideal method for obtaining a discrete representation, mainly because the computational effort to generate all of the efficient extreme points is not reduced. Moreover, the sample of points obtained prior to filtering may not be itself representative of XE. Depending on the geometry of the feasible region, some of the efficient extreme points may be clustered together, while others may be quite distant from one another. In that case, the filtering procedure would generate a sparse set of points, with no representation of the efficient points "inbetween," i.e. the nonextreme efficient solutions. Morse (1980) and Torn (1980) independently suggested using clustering for reducing the size of the efficient extreme points being presented to the DM. They suggest using cluster analysis to construct partitions of the set of efficient extreme points into groups of relatively homogeneous elements. However, all of the disadvantages related to filtering are valid for clustering also. 44 All of these methods are significant in MCDM since they have the goal of portraying the efficient set by a representative subset. However, they fail to be adequate as a solution procedure, or as part of a solution procedure for problem (MMP). Some common disadvantages associated with these methods can be stated as follows. a) The methods concentrate on problem (MOLP). b) The methods generally require as many or more calculations than the procedures developed for obtaining all of the efficient extreme points of X. c) The efficient solutions being presented to the DM still consist of extreme points. Thus, the quality of the representative subset of efficient points obtained may not be satisfactory (cf. Section 1.3). All of these factors suggest that alternative procedures for obtaining representative samples of efficient solutions should be developed. 2.4. Theoretical Background and Notation In this section, we will present some of the background theory that is used in the development of the procedures in this dissertation. These results have been widely utilized in multiple objective mathematical programming. We will also establish some notation that will be used throughout the dissertation. For simplicity, we will restrict our attention to the multiobjective convex programming problem, (MOCP), given by (MOCP) VMAX Cx, s.t. x E X, where X c R" is a compact, convex set, and C is the pxn matrix of objective function coefficients. Let Y = {Cxl x E X}. Then Y is a compact, convex set (Rockafellar 1970). 45 To help analyze problem (MOCP), we will sometimes study a transformed problem in the objective function space which can be written (MOCPY) VMAX Ipy, s.t. yE Y, where I is the p xp identity matrix. Let XE denote the efficient set of problem (MOCP), and let YE denote the efficient set of problem (MOCPY). The following lemma provides the basis for the equivalance of problem (MOCP) and problem (MOCPY). Lemma 2.4.1. (a) For any yo E YE, if x E X satisfies Cx = y, then x E XE. (b) For any x E XE, if y = Cx, then y E YE. Proof. Both (a) and (b) follow easily from the definitions of XE and YE* Let A = {X E RPl entries each equal one, and M is a positive real number. Let Y! = {y E RPIy I y for some y E Y}. For a given y E Y and X E A, we will also need to study the problem (Py,J given by max < XC,x >, s.t. ) Cx y, (2.1) xEX. This problem and variations of it have been widely used in various studies of problem (MMP). The following results establish two properties of problem (Py, J that we will find useful. The proofs follow easily from Theorems 3.1 and 3.2 in Benson (1978). 46 Theorem 2.4.1. Assume that xo E R" and let y = CxO. Then xo belongs to XE if and only if xo is an optimal solution for problem (Py.~ for every X E A. Theorem 2.4.2. If y E Y and X E A, then problem (PyJ) has at least one optimal solution, and any optimal solution xo for problem (Py,x) satisfies xo E XE. Another widelyutilized problem is the "weighted sum" problem (PO max where X E A. Theorem 2.4.3. Suppose that X is a polyhedron. For sufficiently large M, a point x" belongs to XE if and only if there exists at least one X E A such that x is an optimal solution to problem (P) with X = Xo. We will assume henceforth that M is chosen to be large enough to guarantee this property. When X is a polyhedron, it is wellknown that the efficient set for problem MOLP can be described by XE = U {XXX E A} (2.2) where A is a finite subset of A, and for each X A, X, denotes the optimal solution set of the linear program (P) (Yu and Zeleny 1975). Since the optimal solution set of a linear program is a face of its polyhedral constraint set, this implies that XE is a union of a finite number of faces Xx, X E A, of X. However, the representation of XE as given in (2.2) is not generally unique. 47 Another important property of problem (MOLP) is that the extreme points of X that lie in XE comprise a connected set (Yu and Zeleny 1975, Yu 1985), where connectedness is defined in the following sense: Definition 2.4.1. A subset Q of the extreme points of a polyhedron is said to be connected when, for any two elements xo and x' of Q, there is a set of elements x', x2,...',xt of Q such that, for i = 0,1,..,t1, x' is adjacent to xi+. Let Z denote a convex set in R', Q a discrete set in R and let z', z2 E R Let r E R. Then we define the following notation. (int Z) : interior of Z, 9(Z) : boundary of Z, Z, : the set of extreme points of Z, Zi : {z E Rqjz < i for some z E Z}, z' > z2 : z' z2 and z' z2, II z : Euclidean norm of z', rl : absolute value of r, IQ : cardinality of Q. We will assume that z' E Rk is a column vector. We will use the notation (zI)T to denote the transpose of z'. When it is clear from the contents that (z')T is a row vector, we will simply use z' instead of (z')T. CHAPTER 3 OPTIMIZATION OVER THE EFFICIENT SET: TOWARDS AN EXACT ALGORITHM The work of this chapter originates from problem (PD). The problem of minimizing one of the objective functions of problem (MOLP) over the efficient set constitutes a special case of problem (PD). This special case of problem (PD) may significantly aid in finding a discrete representation of XE (cf. Section 1.4). Benson (1991c) proposed a finite bisectionextreme point search algorithm for solving problem (PD). His algorithm requires repeated solutions of certain convex maximization subproblems. This requirement is one of the motivations for the work we will present in this chapter. Another is the need to develop more efficient algorithms for the problem of minimizing a concave function over a polyhedron. In this chapter we present an exact, finite branch and bound algorithm for minimizing a concave function over a compact polyhedron. The algorithm uses a process that we call neighbor generation to ensure implementability and convergence. In Section 3.1, we will describe the algorithm given in Benson (1991c). In Section 3.2, we will present a brief overview of concave minimization algorithms. Section 3.3 will contain the theoretical background required for the development of the new algorithm. The algorithm will be presented in Section 3.4. In Section 3.5, the 49 validity and finiteness of the algorithm will be shown. A discussion of computational benefits and some computational results will follow in Section 3.6. 3.1. The BisectionExtreme Point Search Algorithm In this section, we will give a brief description of the BisectionExtreme Point Search Algorithm of Benson (1991c). A thorough understanding of the Bisection Extreme Point Search Algorithm is not crucial for the development of the concave minimization algorithm that constitutes the major work in this chapter. Therefore, we will only include the portions of the background theory given in Benson (1991c) that are necessary to help motivate the concave minimization algorithm. The reader is referred to Benson (1991c) for details. Let X = {xER" I Ax < b, x 0} be a nonempty compact polyhedron, where A is m n and bER". The following result gives a fundamental property of problem (P). Theorem 3.1.1. Problem (P) has an optimal solution which belongs to X,. Proof. Since X is nonempty and compact, XE is also nonempty and compact (Ecker and Kouada 1975, Yu and Zeleny 1975). Therefore, problem (P) has at least one optimal solution. In addition, from Rockafellar (1970), the compactness of X implies that X has no lines. From Theorem 4.5 in Benson (1984), it follows that problem (P) has at least one optimal solution that belongs to Xx. The BisectionExtreme Point Search Algorithm finds an optimal solution to problem (PD) that belongs to Xx n XE in a finite number of iterations. The algorithm 50 uses the following result to solve problem (PD). The proof of this result can be found in Benson (1991c). Theorem 3.1.2. There exists a positive real number M such that for any M> lM, vd = t', where t' is the smallest value of t E R in the problem (W) given by rt = max s.t. Ax uTA + vdT XTC 0, X >e, x, u,v > 0, such that tr, = 0. Notice that for each value of t, problem (W) is a bilinear programming problem. In the BisectionExtreme Point Search Algorithm, a bisection search is used to find progressively smaller subintervals of the real number line of the form [L,U] that contain vd. For each t E [L,U] chosen during the bisection search, the question whether 7r,>0 or rt=0, where 7r, is as defined in Theorem 3.1.1, must be answered. When r,=0, U is decreased to t. When 7,>0, a local extreme point search procedure is invoked to update L. This procedure finds a new incumbent solution xc E Xex f XE that satisfies the algorithm, at certain times, a termination test is required, which consists of testing whether rL = 0 or rL, > 0. When 7rL = 0, the incumbent solution is an exact optimal solution for problem (PD). 51 Notice that the BisectionExtreme Point Search Algorithm calls for determining whether wrt = 0 or rt > 0 for various values of t found during the search. To accomplish this, a concave minimization algorithm can be used. The next result helps to explain how this can be done. Let t. denote the optimal value of the linear program min Theorem 3.1.3. Let t E R satisfy t > t., and assume that M > max{k,M}, where M is chosen as in Theorem 3.1.2. Then the value of ,i in problem (W) equals the optimal value of problem (V) given by max ht(x), subject to x E X, where ht:RR is the continuous piecewise linear convex function defined by ht(x) = max s.t. uTA + vd XTC 0, X e, u,v >0, for each xER. From Theorem 3.1.3, it follows that to determine whether ir >0 or 7r,=0 for a given t, it suffices to solve problem (V). But problem (VJ is a convex maximization problem, which can be represented as a concave minimization problem. As suggested in Benson (1991c), there are a number of concave minimization algorithms that can be used to solve the subproblems that arise during the implementation of the BisectionExtreme Point Search Algorithm. However, since the main 52 computational burden of the algorithm consists of solving the convex maximization problems, the efficiency of the algorithm to be used is important. It is also crucial that the concave minimization algorithm used to solve the subproblems be finite. Motivated by this fact, we now develop an exact, finite branch and bound algorithm with neighbor generation for minimizing a concave function over a compact polyhedron. Before we proceed with the development of the algorithm, we will present a brief review of the algorithms that have been suggested to solve the concave minimization problem. 3.2. An Overview of Concave Minimization Algorithms The concave minimization problem (CM) can be stated as follows. (CM) min f(x), subject to x EX, where f: D  R is concave on D, and D c R" is an open set that contains X. Let fm. denote the optimal objective function value of problem (CM). Problem (CM) has been the focus of many recent studies. Numerous applications from various fields lead to concave minimization problems. Moreover, several other difficult problems of interest, such as the zeroone integer linear programming problem, and the linear fixed charge problem can be written in the form of problem (CM). For instance, in Section 3.1, we have discussed, in greater detail, how a certain bilinear programming problem can be formulated as a concave minimization over a polyhedron. Although it is well known that problem (CM) has an optimal solution that is an extreme point of X, problem (CM) is a mathematicallydifficult problem that may possess many local optimal solutions. Therefore, special procedures are required for solving it. The algorithms that have been developed to solve problem (CM) are generally based 53 upon cutting planes, extreme point ranking, relaxation, branch and bound searches, and combinations of these approaches. Tuy (1964) proposed a cone splitting algorithm in which, as the algorithm proceeds, portions of the feasible region are eliminated from further consideration by certain cuts. These cuts have come to be known as "Tuy cuts." Later, many authors proposed algorithms that use cutting planes. Among these are algorithms of Zwart (1974), and Majthay and Whinston (1974), which are finite algorithms, and two algorithms of Cabot (1974). One of the algorithms presented in Cabot (1974) uses a composite approach of cutting planes and extreme point ranking. Murty (1968) proposed an extreme point ranking algorithm for solving the linear fixed charge problem. Later, Taha (1973) provided a generalization of this procedure for problem (CM). Falk and Hoffman (1976) presented a relaxation method for solving problem (CM). Their algorithm involves successively minimizing underestimating functions of the objective function over relaxed regions that contain the original feasible region. Other relaxation algorithms are given by Carillo (1977), Rosen (1983), and Falk and Hoffman (1986). The algorithms that incorporate branch and bound search use the idea of partitioning the feasible region into smaller subregions. Then, for each partition element, a lower bound for the minimum of the objective function value over the subregion is computed. The branch and bound procedures may differ in the way the lower bounds are computed, or in the type of partition elements that are used in the branching process. 54 The first of the algorithms that use a branch and bound search was given by Falk and Soland (1969). The algorithm of Falk and Soland requires the objective function to be separable. A modified version of their algorithm is presented in Soland (1974). Both of these algorithms use rectangles as partition elements and are guaranteed to find an exact optimal solution in a finite number of steps. Other typical partition elements that are used in the branching process are simplices (Horst 1976, Benson 1985), and cones (Thoai and Tuy 1980). More information about the methods available for solving problem (CM) can be found in Pardalos and Rosen (1986), Pardalos and Rosen (1987), Horst (1990), Horst and Tuy (1990). The branch and boundneighbor generation algorithm is partly motivated by previous work by Soland (1974), Horst (1976), and Benson (1985). However, significant differences exist, and the convergence of the algorithm is guaranteed in a different way. 3.3. Theoretical Background We will start with the following wellknown definitions, which can be found, for instance, in Horst and Tuy (1990). Definition 3.3.1. Let {vo, v',...,.v} be (n+1) affinely independent points in R". The convex hull of {v, v',...,v"}, denoted CONV{v, v',..,v"}, is called a simplex (n simplex) and the points vO, v',...,v" are called vertices of the simplex. Definition 3.3.2. Let Z c R", I a finite set of indices. A set Q = {Z'I iEI} of subsets of Z is said to be a partition of Z if z = U ', iEl ziNzj = az'nazi v i,jEI, i;j, where azi denotes the (relative) boundary of Z,. Definition 3.3.3. Let S be an nsimplex with vertex set V(S) = {vo, v',...,v'}. Choose a point w E S, w 0 V(S) which is uniquely represented by n n w = Eaiv' ai > 0, i=0,l,..n, Eoi=l, i=0 i=O and for each i such that a, > 0, form the simplex S' obtained from S by replacing the vertex v' by w, i.e., Si = CONV{vO,... ,vi,w,vi ,...,v"}. The set of all simplices S' obtained in this way is called a radial subdivision of S. The algorithm that we will present for solving problem (CM) finds an initial simplex SO that contains X. Throughout the implementation of the algorithm, a collection of subsimplices of So is maintained. The collection of subsimplices of SO is obtained by subdividing SO and subsimplices of SO using radial subdivision at each iteration. By the following result, the collection of subsimplices of SO obtained in this way is a partition of SO. Theorem 3.3.1. The set of subsets S' that can be constructed from an nsimplex S by an arbitrary radial subdivision forms a partition of S into nsimplices. Proof. See Proposition IV. 1 in Horst and Tuy (1990). In the algorithm, for an arbitrary nsimplex S that belongs to the current partition of So, a lower bound has to be computed for f over S n X. To compute this lower 56 bound, the algorithm finds an extreme point of X that minimizes the convex envelope fs of f on S over X, where the convex envelope is defined as follows. Definition 3.3.4. Let Z c R" be a convex, compact set, and assume that h :ZR is lower semicontinuous on Z. A function hz: ZR is called the convex envelope of h on Z when i) hz is convex on Z, ii) hz(x) < h(x) for all x E Z, iii) there is no function q: ZR satisfying i) and ii) such that hz(x) < q(x) for some point x E Z. It is well known that the convex envelope of a concave function on an nsimplex is an affine function and can be obtained by solving a system of linear equations. However, in our procedure for obtaining a lower bound for f over S n X for a given nsimplex S, we will compute the minimum of fs over X, and an extreme point of X that achieves this minimum, without solving a system of linear equations. The following result is crucial in the development of this procedure. For ease of presentation, henceforth we will assume that X is of the form X = {xER" Ax < b}, where  A A = is the (m+n)xn matrix of constraint coefficients and nonnegativity restrictions, and b E Rm"" is given by b = Theorem 3.3.2. Let S c D be any nsimplex with vertices vo,v',...,v", and let fs: S R be the convex envelope of f on S. Consider the linear program (PJ) given by min fs(x), s.t. x E X, and the linear program (P7) given by (P,) min f (vi i=0 S.t. (Av i),i I.b, i=0 i=0 Let r denote the feasible region for the linear program (P,). n n i) If E r, then i = E iiv' belongs to X and fs(x) = if(v i)'. Conversely, i=0 1=0 if n x E X, then there exists a unique j E F such that = iv ', which, furthermore, =0 satisfies fs(i) = Ef(v')i),. i=0 ii) If j is an extreme point of F, then R = E ~ v is an extreme point of X. a=0 i Conversely, if i is an extreme point of X, then the unique j E r such that = V Ci' i=0 is an extreme point of r. iii) The optimal values of problems (P) and problem (P,) are equal. If y7 is an n optimal solution for problem (P), then x = y i' v' is an optimal solution for problem i=0 (Pj. If x* is an optimal solution for problem (Pj, then the unique y' E r such that n x = v i is an optimal solution for problem (P.). i=0 n a  Proof. i) Assume that E r. Let i = Eijivi. Since (Av ')'. < b, the i=0 i=0 definition of i implies that AA < b, i.e., x E X. From Horst and Tuy (1990), fs is an affine function and satisfies fs(v') = f(v'), i=0,1,..,n. Therefore, from the definition of i, fs() = fs( E iivi) = Ei^ifs(vi) = iiff(vi) To show the converse statement in i), assume that i E X. From Definition 3.3.1, the vectors v', i=0,1,..,n, are affinely independent. For each i=0,1,..,n, form the vector w' E Rn+~ whose first n entries equal those of v' and whose (n+ l)st entry is one. Then w', i=0,1,..,n are linearly independent vectors in R".+. Therefore, they form a basis for R"'. Let 9 E R"'+ denote the vector whose first n entries equal these of x and whose (n+1)st entry is one. Then, since 9 E R"+', there exists a unique j E R" such that 9 = iw i. This implies that = .iv i and i,. = 1. Furthermore, since i=O i=0 = i x E X, E (Av ') = A iv' = Ai < b. Therefore, j E I. Finally, since fs is i=0 i=0 a n affine and satisfies fs(v') = f(v'), i=O,l,..,n, we obtain fs(,) = fs( .iv i) = i fs( i) i=0 i=0 = 1=0 ii) Assume that is an extreme point of r. Let = 0 yv i. Then, by part i), i=0 x E X. Since i E X, again using part i), it follows that no E r, d j, exists such that = that x = 7.v' i =0 59 Suppose that i is not an extreme point of X. Then, since 1 E X, for some points x', x2 E X distinct from i and for some a E (0,1), A = ax' + (1a)x2. Since x', x2 E X, by part i), for each j= 1,2, xj = i_9yvi for some 7, 72 E r. Therefore, i=0 n n = (adIv + (1a)y) vi = (Cyi' +(1)e),) 'i n that x = 0iv', this implthat hat E = ay'i+(l1)y2. Therefore, = 1 = 72, since is an extreme point of r. On the other hand, since x' e A, j= 1,2, j = 1,2. This contradiction implies that the assumption that i is not extreme point of X must be false. To show the converse statement in ii), assume that i is an extreme point of X. n Let j be the unique element of r which satisfies i = Ev j . Suppose that j is not an extreme point of r. Then, since j E r, for some points 71, 72 E r distinct from j and for some a E (0,1), = c+(y+(1C)2. Then a = E(t7' +(1c)2)iVi i7,v0 + (la)~fv' i=0 = ax' + (la)x2, n where, for each j= 1,2, xJ = yv i. By part i), xj E X, j= 1,2. Also, since y' j', i=0 j=1,2, and j is the unique element of F satisfying x = i0v i, x 3e j = 1,2. On the other hand, since x is an extreme point of X and i = ax' + (1a)x', i = x' = x2. This contradiction implies that the assumption that j is not an extreme point of r is untenable, and the proof of part ii) is complete. iii) Assume that 7' is an optimal solution for problem (P.). Let x* = ~y'v i and let x be an arbitrary element of X. Then, by part i), x' E X and, for some unique n 7 E r, x = v i. Since y is an optimal solution for problem (P.), i=0 Sf(v )7' f(v )7i (3.1) i=0 i=0 n n From part i), fs(x) = i f(v i) and fs(x) = f(v i). Therefore, from (3.1), i=0 i=0O fs(x') s fs(i). This implies that x' is an optimal solution for problem (P). Also, since fs(x') = [ f(v ')*, the optimal values of problems (P) and (P.) are equal. i=O Assume that x' is an optimal solution for problem (PJ. Let 7' E I be the unique n element of r satisfying x' = v i. Let be an arbitrary element of r. Then, by i=0 a part i), x = i v belongs to X. Since x' is an optimal solution for problem (PJ, this i=0 n n implies that fs(x') fs(x). From part i), fs(x') = f(v ') and fs(x) = f(v)7. i=0 i= n n Therefore, f(v')y,* < f(v i) This implies that 7" is an optimal solution for i=0 i =0 problem (P,), and the proof is complete. 61 From Theorem 3.3.2, the optimal values of problems (P,) and (P7) are equal and for any optimal extreme point optimal solution y E r for problem (P.), x' = y 'v i E X is an extreme point optimal solution for problem (Pj. Using these i=0 results, the algorithm will repeatedly solve problem (P,) instead of problem (Pj to find lower bounds and extreme points of X which achieve these lower bounds. 3.4. The Concave Minimization Algorithm At the beginning of the algorithm, an initial simplex SO that contains X is constructed. Throughout the algorithm, at any step k, a lower bound LBk of f,. must be computed. To aid in this computation, first, a partition Qk of SO is obtained. To accomplish this, the algorithm performs a radial subdivision of a subsimplex Sk" of So to yield a partition Tk of S'. Then Qk is obtained by removing Sk' from QO1 and adding the elements of T", where Q"' is the partition of SO that is available from step k1. Also, for each nsimplex Si in Q" that belongs to Tk, a lower bound wj for the minimum of f over Sn X is computed. For the remaining elements Sj of Q", a lower bound wj for the minimum of f over Si n X is available from some previous step. These lower bounds are computed by solving problems of the form (P,). To find the global lower bound LBk for the minimum of f over X, the algorithm computes the minimum wj of the values wj, j E I(Qk) and sets LBk = wl, where I(Qk) denotes the finite index set of the elements in Qk. In combination with the branch and bound scheme, the algorithm uses a neighbor generation process, primarily to ensure implementability and convergence. However, 62 the incorporation of the neighbor search process also provides a greater potential for finding better incumbent solutions. In each iteration of the algorithm, an extreme point x of X is found, which is a candidate for serving as a basis of a radial subdivision of some subsimplex that is in Qk. The neighbor generation process is invoked for each such point x in order to search among all the neighboring extreme points of x in X to find the ones that have not been used as vertices of any subsimplex in Qk. A list L is kept of all such extreme points of X obtained in this manner. Each time step k is executed, unless the algorithm terminates, an extreme point x of X must be found to serve as a basis for a radial subdivision in the next step. To do this, the algorithm proceeds as follows. In finding LBk, the algorithm determines the lowest lower bound wj that is available in step k. The extreme point xj that is found by solving the problem (P,) used to help compute wj is inspected to see if it has served as a vertex of any of the subsimplices of SO that are in the current partition. If it has not, the algorithm chooses x` as the extreme point of X to be used as a basis for radial subdivision of a simplex that it belongs to. However, if xJ is a vertex of a simplex that is in the current partition, then, the algorithm examines the list L. If the list L is empty, then the algorithm terminates with the incumbent solution as an optimal solution for problem (CM) (see Section 3.5 for the validity of this approach). If the list L is nonempty, the algorithm searches among the extreme points in the list L in order to find one that fails a fathoming test. During this search, points are removed from L as they are examined, and the neighbor generation process is invoked for each such point. If L 63 becomes empty as these points are examined, the algorithm terminates, and the current incumbent solution is an optimal solution for problem (CM). Thus, in each step k, the algorithm either successfully identifies an extreme point of X as the basis of a radial subdivision process, or terminates with the list L being empty. For each point that serves as a basis for radial subdivision, the neighbor generation process is invoked regardless of the way it was obtained. The extreme points that serve, or are considered for serving, as a basis for radial subdivision are placed in another list VL for bookkeeping purposes. Throughout the algorithm, many feasible solutions are found for problem (CM). Therefore, the upper bound, which is the minimum of the objective function values in problem (CM) of the available feasible points, is updated as these points are obtained. The incumbent solution, denoted x' in the algorithm statement, is a feasible point that achieves this minimum. The Algorithm Statement Step 0. Step 0.1. Choose an nsimplex So c D such that SO 2 X, and let Qo = {SO}. Let the vertices of SO be { vo, v',...,v" }. Set N = 0, and VL = { vo, v',...,v" }. For each i E {0,1,...,n) for which v' E X, set N = N U { v }, and generate the set E, of extreme points of X adjacent to v' in X which do not belong to VL. For each n i E {0,1,..,n} for which v' ( X, set E, = 0. Set L = UEi. If N = 0, set i=0 UB = +oo and go to Step 0.2. IfN 0, choose xc E argmin { f(x)l x E NUL } and set UB = f(x'). 64 Step 0.2. Find the optimal value qo and an extreme point optimal solution ' E r for the linear program (P.) (see Section 3.3 for the definition of problem (P,)). Let n x = ~y 7, vI, set wo = q0, and set x0 = x'. Generate the set E of extreme points in i=U X adjacent to xo which do not belong to VL. If E c L, go to Step 0.3. Otherwise, set L = LU E, and continue. Step 0.3. Set LBo = w0. Set UBo = min { UB, {f(x) xE{x}UE} }. If UB 4 +oo, choose x E argmin { f(x), {f(x) xE6{P}UE }} and set x = i. Otherwise, choose x E argmin {f(x) x E {o} UE} and set xc = i. If LBo = UBo, conclude that xc is an optimal solution for problem (CM) and stop. Otherwise, set k = 1 and go to step k. Step k, k> 1. At the beginning of step k, a partition Qk' of So is available from the previous step. Also available for each nsimplex Sj E Qk' are a lower bound wj for the minimum of f over Sn X and an extreme point xj E X found in the process of computing this lower bound. Assume that Sk' E Q1 denotes an element of Qk' which contains the point i' computed in the previous step. Step k.1. If ' C L, remove "'' from L. Using R~' E Sk' as a basis, perform a radial subdivision of S'' to obtain a partition Tk of Sk'. Set VL = VL U {x1'}. Step k.2. For each nsimplex Sj E Tk: (i) With v', i=0,1,.. ,n set equal to the vertices of S, find the optimal value qj and an extreme point optimal solution y' E P for the linear program (P,); n (ii) Set x = y 7, v i; and (iii) Set wj = max {q1, wk1). 65 Step k.3. Set UB, = min {UBI,, {f(xj) Si E T}}. If UB, = f(x) for some j such that Sj E T, set x' = xi. Step k.4. Set Qk = (Qk\Sk) U T1. Step k.5. Let wj = min { wjI j E I(Qk)}, and set LBk = wj. Step k.6. If LB, = UB,, conclude that x' is an optimal solution for problem (CM) and stop. Otherwise, continue. Step k.7. (i) If x VL, set il = x1 and go to Step k.8. Otherwise, continue. (ii) If L = 0, conclude that xc is an optimal solution for problem (CM) and stop. Otherwise, remove any point x from L and continue. (iii) Find any nsimplex SJ E Qk which contains x. If w UBk, set = x and go to Step k.8. Otherwise, continue. (iv) Generate the set E of extreme points of X adjacent to x which do not belong to VL. Set VL = VLU {x}. If E c L, go to Step k.7(iii). Otherwise, set L = LUE, find x E argmin {f(x'), { f(x)I xeE}}, set x' = i, set UBk = f(xc), and go to Step k.7(ii). Step k.8. Generate the set E of extreme points of X adjacent to j which do not belong to VL. If E c L, set k = k+l and go to Step k. Otherwise, set L = LUE, find i E argmin {f(xc), {f(x)l xEE}}, set x' = x, set UBk = f(xc), set k = k+l, and go to Step k. The set N in Step 0.1 stores each vertex v' of SO which is also an extreme point of X. Since SO Q X, it follows that for any vertex v' of S, if v' E X, then v' is an extreme point of X. 66 In Step k.7(iii), if the lower bound wj for the simplex Sj containing the point x removed from L exceeds UBI, then control passes to Step k.7(iv). In Step k.7(iv), the usual neighbor generation process for x and the associated incumbent update are performed, and x is added to the list VL. However, control then passes to Step k.7(ii) rather than to Step k+1. In particular, no radial subdivision of the simplex S' is performed, and no linear programming problems of the form (P,) that would be associated with this radial subdivision are solved. In this sense, the simplex S is fathomed. Notice that the point x is not used as a basis of a radial subdivision in the next step. However, since it was a candidate for such a use, x is added to the list VL. 3.5. Validity and Finiteness We will now show that the algorithm finds an optimal extreme point solution for problem (CM) in a finite number of steps. To show this, the following result is needed. Theorem 3.5.1. For any k> 1, if L = 0 in Step k.7(ii) of the algorithm, then VL contains every extreme point of X. Proof. Assume that k> 1. To prove the theorem, we will prove the contrapositive. Therefore, assume at Step k.7(ii) that an extreme point x E X exists which does not belong to VL. Then either (1) at least one extreme point of X adjacent to i belongs to VL or (2) no extreme point of X adjacent to i belongs to VL. Case 1. At least one extreme point x of X adjacent to i belongs to VL. For any extreme point y of X added to VL thorough Step k.7.(ii) of the algorithm, from Steps 0.1 and 0.2 and Steps w.1, w.7, and w.8, 1 < w k, the neighbor generation process guarantees that all extreme points of X adjacent to y which do not belong to VL are 67 contained in L. Since A is an extreme point of X adjacent to i which belongs to VL, and x 0 VL, this implies that x E L. Hence, in this case, L # 0. Case 2. No extreme point of X adjacent to i belongs to VL. Consider the extreme point xO E VL. Since X is a polyhedron, there exist a finite number of extreme points y, y2,.., y' of X such that R is adjacent to y', yh is adjacent to yh+" for each h = 1,2,..,t1, and y' is adjacent to i0. Let Y'= {y E {y',y2,..,y'}I y1VL}. Since y' is adjacent to i and no extreme point of X adjacent to a belongs to VL, y' ( VL. Therefore Y' 0 and we may choose an integer h' such that h' = max { hE {1,2,..,t}) yh < VL}. If h' < t, then yh' is adjacent to yb'+1 E VL. If h* = t, then yb' is adjacent to x0 E VL. In either case, yh' is an extreme point of X which does not belong to VL but which is adjacent to an extreme point of X which does belong to VL. Then, by the same reasoning as used in Case 1 for x, it follows that yb* E L. Since this implies that L ; 0, the proof is complete. U We may now show the following result. Theorem 3.5.2. Whenever the algorithm terminates, the algorithm's current incumbent solution x' is an extreme point optimal solution for problem (CM). Proof. Consider any step k, k>O of the algorithm. From Step 0.3 or, for k> 1, from Step k.5, LBk = min {wj jE I(Q)}. (3.2) For each j E I(Qk), by Theorem 3.3.2, qj, calculated in Step k.2(i) for some k < k, is a lower bound for the minimum of f over X f Sj. In addition, for any sets A and B such 68 that A c B, any lower bound for the minimum of f over B is a lower bound for the minimum of f over A. The latter two statements imply that for each j E I(Q"), wj, calculated in Step k.2(iii) for some k < k, is a lower bound for the minimum fQ, of f over XnSi. Therefore, min {wjl j E I(Q)} < min {fI J E I(Q0)}. (3.3) By Theorem 3.3.1, {S I j E I(Qk)} is a partition of SO Q2 X. This implies that the right handside of the inequality (3.3) is identical to f.. From (3.2), this implies that LBk < f. Suppose that the algorithm terminates in Step k. Then either (1) LB, = UB, or (2)L = 0. Case 1. LBk = UBk. Since UB, = f(xc), where xc is the current incumbent extreme point solution, it follows that in this case, LB, = f(xc). From the discussion above, LB, < f... Therefore f(xc) < f... Since x" E X, this implies that f(xC) = f, so that x' is an extreme point optimal solution for problem (CM). Case 2. L = 0. Then k 1, the algorithm terminates in Step k.7(ii), and, from Theorem 3.5.1, VL contains every extreme point of X. From Steps 0.1, 0.3 and Steps k. k.3, k.7, and k.8, 1 k < k, for any extreme point i of X contained in VL by Step k.7, UBN 5 f(i). Since UBk = f(x), where x' is the current incumbent extreme point solution, and since VL contains every extreme point of X, this implies that f(xc) < f(x) for all extreme points x of X. Since problem (CM) has an optimal solution which is an extreme point of X, it follows that x' is an extreme point optimal solution for problem (CM) and the proof is complete. The algorithm always eventually terminates by the following result. Theorem 3.5.3. The algorithm terminates in a finite number of steps. Proof. Suppose, to the contrary, that the algorithm does not terminate in a finite number of steps. Then Step k.7 is executed an infinite number of times. Therefore, in an infinite number of executions of Step k.7, either (1) the point x' in Step k.7(i) does not lie in VL, or (2) a point x in Step k.7(ii) is removed from L. Case 1. In an infinite number of executions of Step k.7, xj in Step k.7(i) does not lie in VL. From Steps k. and k.7(i), whenever x, 1 VL in some step, it is added to the set VL in the next step. But each point xJ is an extreme point of X, and X contains a finite number of extreme points. The latter two statements imply that it is impossible for xJ not to belong to VL in an infinite number of executions of Step k.7(i). Therefore, this case cannot occur. Case 2. In an infinite number of executions of Step k.7, a point x in Step k.7(ii) is removed from L. From Steps 0.1, 0.2, k.7 and k.8, L contains only extreme points of X, and the only points ever added to L by the algorithm are extreme points of X not already contained in L. Since X contains a finite number of extreme points, this implies that it is impossible to execute the removal of a point x in Step k.7(ii) from L an infinite number of times. Therefore, this case cannot occur. It follows that the assumption that the algorithm does not terminate in a finite number of steps is false, and the proof is complete. Taken together, Theorems 3.5.2 and 3.5.3 imply that the algorithm is guaranteed to find an extreme point optimal solution for problem (CM) in a finite number of steps. 3.6. Computational Benefits and Preliminary Computational Results Computational benefits. The branch and boundneighbor generation algorithm finds an exact extreme point optimal solution for problem (CM) in a finite number of iterations. The other major advantages of the algorithm are as follows. a) The objective function f is required to be concave on an open set D containing X. No further conditions are imposed on f. In particular, f need not be separable or even analytically defined. b) The algorithm requires no nonlinear computations and no determinations of convex envelopes or other underestimating functions. c) The linear programming problems solved during the branch and bound search do not grow in size, and they differ from one another in only one column of data. d) The fathoming test in Step k.7(iii) helps avoid unnecessary computations. In particular, when w > UBk, the nsimplex Sj is not partitioned and no linear programs of the form (P,) that would be associated with this partitioning process are solved. Notice that the linear programming problems solved during the execution of the algorithm are all of the form of problem (P,). Thus they all contain (m+n+l) constraints and (n+1) variables. Furthermore, for each nsimplex Sj E I* in step k.2, the vertices of SJ are identical to those of its parent nsimplex Sk1, except that in Si the vertex xi' replaces one of the vertices v' of S"'. As a result, the linear program (P,) associated with Si which is solved in step k.2 differs from the one solved earlier for Sk' only in the coefficient for y in the objective function and in the first m constraints. This suggests that if the simplex method is used, for instance, to solve the linear programs 71 (PR), the optimal basis associated with S"' can be used as a starting basis for the simplex method solution of the problem (P,) associated with Sj. In this way, the linear programs (P) can, in general, be solved more efficiently than they would be by using the traditional Phase IPhase II approach of the simplex method (Murty 1983). Computational results. We have written a computer code in C which implements the proposed algorithm. We constructed 60 test problems, and used the code to solve these problems on an IBM 3090 Model 600J mainframe computer. In this section, we describe the code, the sources of the test problems, and the computational effort that the code required to solve the problems. It has been pointed out (Benson and Erenguc 1990, Horst, Thoai and Benson 1991, Pardalos and Rosen 1987) that the difficulty of finding a globally optimal solution to a concave minimization problem limits the sizes of problems solvable by reasonable effort. Furthermore, the amount of reported information about the computational behavior of existing algorithms is limited. Since there are no generally accepted test problems, we have constructed various test problems partially based on some data in the literature. The goals of the computational experiments reported here were limited to making some preliminary conclusions about the algorithm. The code uses the simplexbased subroutines of the Optimization Subroutine Library (IBM 1990) to solve the linear programming problems of type (P) called for in the algorithm. In each run, the initial simplex SO that contains X was constructed separately using the method proposed by Horst (1976). The linear programming problems required by Horst's method are also solved by the simplex method procedures 72 given in the subroutines of the Optimization Subroutine Library (IBM 1990). The branch and bound tree and the lists L and VL were maintained and processed using dynamic data structures. The test problems were constructed as follows. Using data from Pegden and Petersen (1979), we constructed eight nonempty, compact polyhedra of five different sizes, where size is defined by the number of rows (m) and the number of columns (n) in the matrix A. Next, we obtained six different types of concave functions from the literature to use as objective functions. The forms and sources of these functions are given in Table 3.1. In function number 4, K represents any positive integer. We also obtained some variations of these functions by appending linear terms, negative quadratic terms, or both. Last, five categories of 12 test problems each were constructed by combining the eight compact polyhedra with the six different types of objective functions in various combinations. For simplicity, we solved only problems in which each extreme point of X that was encountered was nondegenerate. This shortened the code required to execute the neighbor generation process. This part of the code uses simplextype pivots, which we call pseudopivots. Although the degenerate case can also be handled by pseudopivots, the process in this case can become rather complicated (Murty 1968, Murty 1983). For this reason, in these initial computational experiments, we decided to eliminate the problems in which degenerate extreme points were encountered. Table 3.1. Some objective function forms function no. functional form 1 Ix, + 1) xj j=2 J 2 [1 + Ejx2] j=1 3 i Ex lIn[l+ Ex ] j=1J j=l1 j a nI 4 K Ex +2(x x ) j=1 j=1 5 129x2 +242x x2 129x,2 + 1258x, 1242x, n a 6 ( E x2) In( j2) j=1 j=1 source Horst, Thoai and Benson(1991) Horst, Thoaiand Benson(1991) Horst, ThoaiandBenson(1991) Konno (1976) Tuy, Thieu and Thai(1985) Horst, Thoai and Benson(1991) Each run was terminated when either an incumbent solution with an objective function value within five percent of the optimum was found, or the list L became empty. We used the following measures as evaluation criteria. 1) Number of iterations, 2) number of nodes created in the branch and bound search (which equals the number of linear programming problems solved), 3) total number of simplex iterations performed, 4) total number of pseudopivots performed, 5) CPU time in seconds. Table 3.2 gives the averages of these measures by category. In Table 3.3, the minimum and the maximum for each measure in each category are reported. A more 74 detailed description of the individual test problems and computational results is given in the Appendix. Table 3.2. Computational results: averages m n Iterations Nodes Simplex Pseudo CPU time ___ Iterations pivots Table 3.3. Computational results: extremes m n Iterations Nodes Simplex Pseudo CPU time Iterations pivots From Table 3.2, it can be observed that the computational effort required to solve the test problems does not necessarily monotonically increase as m or n increases. Also, from Table 3.3, it can be noted that the variance in the computational effort needed to solve the problems within a category can be relatively small or large. These observations indicate that structural factors other than problem size have significant effects on the computational behavior of the algorithm. This conclusion is consistent with 75 computational results reported for some other global optimization problems (see, for instance, Benson and Erenguc 1991, and Horst, Thoai and Benson 1990). Although our computational results seem to be encouraging, generalizations about the computational behavior of the algorithm would require further experimentation. CHAPTER 4 OPTIMIZING OVER THE EFFICIENT SET: FOUR SPECIAL CASES AND A FACE SEARCH HEURISTIC ALGORITHM In this chapter, we study four special cases of problem (P). We present procedures for detecting and solving these special instances of the problem. We also present a heuristic algorithm for solving the general problem. First, preliminaries are given. The theoretical background required to analyze the special cases and the development of the heuristic algorithm are given in Section 4.2. In Section 4.3, we present the special cases, and give detection and solution procedures for each case. The face search heuristic algorithm is given in Section 4.4. A discussion follows in Section 4.5. 4.1. Preliminaries Let X = {x E R1 Ax < b}, where A is mxn, b E Rm. Assume that X is nonempty and compact. Then, from Rockafellar (1970), Y is a nonempty, compact polyhedron. To help analyze problem (P), we will be interested in the linear counterpart of problem (MOCPY) which we will denote problem (MOLPY). The concept of the ideal solution for problem (MOLPY) is important in analyzing both problems (MOLP) and (MOLPY). Let y' denote the ideal solution for problem (MOLPY). Notice, from Definition 1.1.2, that y' need not belong to Y. However, when y' E Y, then YE = {y'}. Thus, it is clear that the case of y' belonging to Y is a special case of problem (MOLPY). 77 Another special case for problem (MOLPY) is given in the following definition. Definition 4.1.1. Problem (MOLPY) is said to be completely efficient when Y = Y.E Similarly, when X = XE, problem (MOLP) is called completely efficient. Notice that problem (MOLPY) is completely efficient if and only if problem (MOLP) is completely efficient. To help analyze problem (P), we will also be interested in the linear programming relaxation of problem (P). This problem, which we will denote problem (LPuR), is given by (LPua) max Let UB denote the optimal objective function value of problem (LPu ). Then UB is a finite number and gives an upper bound on the optimal objective function value vd of problem (P). Since X and Y are each nonempty and compact, from Rockafellar (1970), X,, and Yx are each nonempty. We can now describe the four sets of conditions which give rise to the four special cases of problem (P) of concern in this chapter. Case 1: complete efficiency. In this case, problem (MOLPY) is completely efficient. From an earlier observation, this case can be equivalently defined as the case when problem (MOLP) is completely efficient. Although the frequency of this case is unknown, it may be more common than was once thought, especially when X has no interior (Benson 1991a). 78 Case 2: ideal solution with linear dependence. In this case, the ideal solution y' for problem (MOLPY) belongs to Y, and the problem of interest is the special case (PD) of problem (P). It is rare for the condition y' E Y to hold. On the other hand, problem (PD) may arise in certain common situations. For instance, minimization of a criterion function < c,x> of problem (MOLP) over XE constitutes a special case of problem (PD). Case 3: bicriterion case with linear dependence. In this case, p = 2 and the problem of interest is the special case (PD) of problem (P). When p = 2, problem (MOLP) is called a bicriterion linear programming problem. Such problems are not uncommon and have received special attention in the literature (see, for instance, Geoffrion 1967, Wendell and Lee 1977, Aneja and Nair 1979, Benson 1979). Case 4: relaxation case. In this case an optimal solution x* E Xx to the linear programming relaxation (LPus) of problem (P) exists which belongs to X,. In Section 4.3, for each case, we will describe a simple linear programming procedure which detects the case and solves problem (P) under the special conditions of the case. To develop the heuristic procedure, we will partially rely on the geometrical background and the mathematical programming problems provided in Section 2.4. Recall, from Section 2.4, that XE = U {XIX E A} where, A c A is a finite set, and for each X E A, X, denotes the optimal solution set of the linear program (P)J (Yu and Zeleny 1975). Since the optimal solution set of a 79 linear program is a face of its polyhedral constraint set, this implies that XE can be represented as a finite union of the faces Xx, X E A, of X. However, this representation is generally not unique. To aid in its search, the heuristic procedure will solve the linear program (Py,j for various values of y E Ys and X E A. In order to determine the particular points from Y to be used in the heuristic algorithm, we will make use both the ideal point of Y and the antiideal point of Y, where the latter is defined as follows. Definition 4.1.2. The point yA' E R' is called the antiideal point of Y when, for each AI j = 1,2,...,p, y minimizes y, over Y. The following two definitions pertain to redundancy in linear programming, and can be found, for instance in Karwan et al. (1983). These definitions will be used in the proof of Lemma 4.2.2, which is required in the analysis of the third special case. First, we need to establish some notation. Since, in the third special case, problem (MOLP) is assumed to be a bicriterion linear programming problem, we will assume, only for the purposes of these definitions, that Y is given by Y = { yER2j My < u, y O0 }, (4.1) where M is a kx2 matrix and u E R'. Also, let 2 Yr a {yER2J 1 mjj j=1 (i.e. delete the r* constraint in the definition of Y), where m1j refers to the entry of M in the i" row and j" column, and let 2 s,(y) = ur E myj for each r E {1,...,k} and y E R2. j=1 2 Definition 4.1.3. The constraint E m~yi u, is redundant in (4.1) if and only if j=1 s, & min{sr(y) yEY } > 0. 2 Definition 4.1.4. The constraint E myj 5 u, is weaklyredundant (wredundant for j=1 short) if and only if it is redundant and 9, = 0. We are now ready to give the theoretical background for the four special cases of problem (P) that we have described and for the heuristic algorithm that will be presented in Section 4.4. 4.2. Theoretical Background Recall, from Chapter 3, that problem (P) has an optimal solution that belongs to Xe. This result will be utilized in the procedures of this chapter. To help develop the remaining results of this section, we will use the following technical properties. Lemma 4.2.1. For any yo E Ye, there exists a point x E X,, such that Cxo = y. Proof. Suppose, to the contrary, that if Cx = y for some x E X, then x ( Xe. Choose any x E X such that Cx = yo. Then, by assumption, x ( Xx. Since X is compact, this implies that there exist points z',z2,...,zq E Xx and scalars a1,a2,...,aq > 0 such that q q x = z, a, = 1, i=I i=1 where q > 2 (Rockafellar 1970). Since Cx = yo, this implies that = (4.2) y0 = E oly i, (4.2) i=1 81 where, for each i = 1,2,...,q, y' = Cz' E Y. For each i = 1,2,...q, by assumption, since z' E XX, Cz' C y must hold. Therefore, y' : y, i = 1,2,...,q. Let Y\{y0} = {y E YIy y}. Then it is easy to see that since y is an extreme point of Y, Y\{y} is a convex set. For each i = 1,2,...,q, since y' E Y and y' yO, it follows that y' E Y\(y}. Therefore, any convex combination of the points y', i = 1,2,...,q must lie in Y\{y}. But since Y is a convex set and the scalars a,, i = 1,2,...,q are positive and sum to one, this contradicts (4.2), and the proof is complete. 0 The next result concerns the special conditions described in Case 2 in Section 4.1. Theorem 4.2.1. Assume that the ideal solution y' for problem (MOLPY) belongs to Y, and that d is linearly dependent on the rows of C. Then any x E X, is an optimal solution for problem (PD). Proof. Since d is linearly dependent on the rows of C, we may choose a vector w E R' such that dT = wTC. (4.3) Consider the problem (PY) given by max < w,y >, subject to y E YE* By applying Theorem 3.1.1 to problem (PY), it follows that problem (PY) has an optimal solution which belongs to Y e n YE. On the other hand, as mentioned earlier, since y' E Y, YE = {yJ}. Therefore, y' E Y.e and y' is an optimal solution for problem (PY). 82 Choose any x E XE, and let y = Cx. Then, by Lemma 2.4. l(b), y E Y. Recalling that YE = {y'} whenever y' E Y, this implies that y = y'. Therefore, Cx = y'. This implies, using (4.3), that = Since point x E XE is an optimal solution for problem (PD). 0 Remark 4.2.1. Notice in the proof of Theorem 4.2.1 that the point x E X. satisfies Cx = y'. It follows that under the conditions of Theorem 4.2.1, any optimal solution x E XE for problem (PD) satisfies Cx = y'. Remark 4.2.2. The assumption in Theorem 4.2.1 that d is linearly dependent on the rows of C cannot be omitted, as shown by the following example. Example 4.2.1. Let X = {x E R310 < xj 1, j = 1,2,3}, let p = 2, and let C and d be given by 10 0i] C = and d = , C= 1 0 respectively. Then Y = {(yl,Y2) E R210 < yi 1, i = 1,2}, and (y')T = (1,1) E Y. The point (xo)T = (1,1,0) satisfies x E XE. However, since XE = {x E R31x, = 1, x2 = 1, 0 5 x3 < 1}, the unique optimal solution for problem (P) is (x')T = (1,1,1). Therefore, the application of Theorem 4.2.1 to this example is invalid and yields the 83 erroneous conclusion that (xo)T = (1,1,0) is an optimal solution for problem (P). This is because d is not linearly dependent on the rows of C in this example. The next result concerns the special conditions of Case 3 given in Section 4.2. Before proving this result, we need the following lemma. Lemma 4.2.2. Let Y c R2 be a nonempty polyhedral set defined as in (4.1). Let yO > 0 be a degenerate extreme point of Y, and assume that exactly two distinct edges E1 and E emanate from yo. Let Bo = { BO,...,Bo}, where v > 1, be the set of bases that correspond to yo. Then there exists v' E {1,...,v} such that in the simplex tableau that corresponds to the basis B,., there exist two distinct nonbasic variables s,, s,2 such that E, and E2 are obtained by increasing s, and sN2, respectively, to a positive level. Proof. Throughout the proof, we will use the following notation. mi : the i" row of M, i = ,..,k, Hi = { yER21 S' = { yER21 Sk+'= { yR2 y, > 0 }, i = 1,2. We will assume without loss of generality that mi 4 0, i= 1,..,k. k+2 Also, note that y = S'i Let Y = {(y,s)ER+21" My + Ik = u, y,s>0 }, where Ik is the kxk identity matrix, and s = (sl,..,s)T E Rk is the vector of slack variables in the constraints of Y. Let yo be a degenerate extreme point of Y, and let E, and E2 denote the two distinct edges that emanate from yo. For ease of notation, assume that E, c H' and E2 c H2 (notice that this can be done by relabeling the constraints of Y if necessary). 84 Choose a basic feasible solution B,. corresponding to yO with s, and s2 as nonbasic variables. Let s3,..,s, denote the basic variables that are at zero level at yo. Then the following hold: (i) The structural variables y, and Y2 are basic at a positive level since yO > 0. Thus (k2) slack variables are basic. (ii) Since yo is a degenerate extreme point, t 3 holds. (iii) Exactly two slack variables are nonbasic, namely s, and s2, (t2) are basic at a zero level, and (kt) are basic at a positive level. The organization of this proof is as follows. First, we will state and prove three claims. The second claim will establish the fact that the vectors m, and m2 are linearly independent. Then using this fact and the third claim, we will show that, for any r E {3,..,t}, the rT constraint is wredundant in the definition of Y. This in turn will be used to complete the proof. Claim 1. H' and H2 are distinct hyperplanes. Proof of Claim 1. Suppose, to the contrary, that H' = H2. Since m, 6 0, it follows that dim(H') = 1 and there exists a point d' E H' such that d' and yO are affinely independent. Then d' ; yO and any y E H' is an affine combination of yO and d'. Thus, for any y E H', y = Xd' + (lX)yo for some X E R. Equivalently, for any y E H', y = yo + X(d'yO) for some E R. Let y' E E, be a point such that y' y yO. Since E, c H', y' = yo+X,(d'yo) for some X, E R, X, 40. Let y2 E E2 be a point such that y2 y0. Similarly, since H' H2, y2 = y0+X2(d'y) for some X2 E R, X2 0. Now, let X = min{ t IX, IX1 }. Then 85 , is positive. Furthermore, X, and X2 are opposite in sign. To see the latter statement, suppose, to the contrary, that hX and X2 are both positive. Then y3 a y0+X(d'yo) E EnE2, since both E, and E Eare convex. Since EnE,2 = {yO}, y3=y0. But since X>0, and d' ; yO, y3y0, which is a contradiction. A similar argument follows when X, and X2 are both assumed to be negative. Let y1 = yo+( )X(dly) and y2 = yO+( X__)(dlyo). Then' E E and IxiI IX2 '2 E Ez. Also, S'l yO and 92 yO since X,, X2 are nonzero, is positive, and d' yO. Furthermore, I 1. 2 = 1 ^+( )X(d'y0))+ 1 )X(dly0)) 2 2 2 2 X2 2 = yO + ( + )(dI'y) 2 IX,I KI =yO + 2(0) (d'y0) = y0 where the third equality follows because X, and X2 are opposite in sign. But this result contradicts the fact that yo is an extreme point of Y. So our assumption is false and Claim 1 holds. Claim 2. The vectors m, and m2 are linearly independent. Proof of Claim 2. Assume, to the contrary, that m, and m2 are not linearly independent. Then, for some X, E R, X, ;0, m, = Xm2 is true. Since yO E H' and yO E H2, U1 = H' = yER2 R1 = {y E R2 m X = {y E R21 which contradicts Claim 1. Notice that the third equality above follows because X, e0. Thus, Claim 2 holds. Claim 3. dim(Y) = 2. Proof of Claim 3. Let y' E E,, and y2 E EFbe two points in Y such that y' yO, and y2;yO. Then y' = yo + Xmt for some mi E R2, mi 0, where m' ; 0, where are linearly independent, it follows that m' and mi are also linearly independent. To see this, suppose, to the contrary, that mi and mi are linearly dependent. Then mf = Xm' for some X E R, X0. So, we have 0 = < m2, m, > = 0, since X? 0. We also have, by definition of m', that < mi, m7> =0. Hence (m)T =0, where mi 0. So, columns of the square matrix i are mq m2 linearly dependent, which in turn implies that m, and m2 are linearly dependent. Since this contradicts Claim 2, it follows that mi and mi are linearly independent. Now, let d' = y'y0 = Xmt, and d2 = y2_y0 = X2m'. Then d' and d2 are linearly independent. Hence, yO, y', y2 are affinely independent. Thus dim(Y) > 2. Since Y c R2, dim(Y) 2. Therefore dim(Y) = 2, and the proof of Claim 3 is complete. 87 Since, by Claim 2, m, and m2 E R2 are linearly independent, they form a basis for R2. Hence for any p E R2, p = Xim, + X2m2 for some X,, X2 E R. Now, let r E {3,..,t}. Then m, = XIm, + X2m2. (4.5) Case 0. X, = 0, X2 = 0. Then, from (4.5), m, = 0, which is a contradiction. So, Case 0 cannot hold. Case 1. Exactly one of X, and X2 is zero. Assume without loss of generality that X, = 0, and X2 0. Then, by (4.5), m,= X2m2, (4.6) and u, = Since, by definition, S' = {y E R21 S' = {y E R21 X2 Case la. X2 > 0. From (4.8), X2 > 0 implies that S' = {y E R21 k+2 k+2 Y = nsi = n Si = y. From Definition 4.1.3, this implies that constraint r is ii==l,ir redundant in (4.1). Furthermore, since Y = Y, and yo E Y, yO E Y,. Hence 5, = 0. Therefore constraint r is wredundant. Case lb. X2 < 0. From (4.8), since X2 < 0, it follows that S' = {y E R21 S' n S2=H2. Thus Y c H2, which contradicts the fact that dim(Y)=2 (see Claim 3), since dim(HI)= 1. Therefore, Case lb cannot hold. Case 2. X, ; 0, X2 O. Then, by (4.5), m, = X1m, + X2m2, and u, = = X < mi,y> +X2 u, = X1U1 + X2u2. (4.9) Case 2a. X1 and X2 have opposite signs. Assume without loss of generality that X, < 0, and X2 > 0. Let y' = yo + elm where m 0, sufficiently small to guarantee that y, E Y. Note that e, exists since E_ c H2, and E2 c Y. Without loss of generality, assume that E, > 0 (notice that when e, >0 cannot hold for a given m', then by choosing m"' = m', this assumption can be satisfied). Since yi E Y, we have S m= < m,m: > < 0, i= ,..,t. (4.10) Notice that (4.10) follows since Also, from (4.5), < m,, y > = X < m,,y' > + X2 m2,Y >. This implies that = Xlui + X2u2 + EIXl From (4.10), with i=l, we know that Case 2ai. From (4.11) and (4.9), it follows that choice of y, y5 E H2. Recall that yO E H2 nH'. Thus yO, y' E H2nHr. Also, since E~ > 0and m' 0, y1 ; yO. Since 1, yO E R2'and yO, dim(H nAH) > 1. But dim(H2n H') 5 dim(H2) = 1. Thus, dim(H2 HF) = 1. SoH2 = H'. Hence, Y, = Y, so that by Definition 4.1.3, constraint r is redundant in (4.1). Since, in addition, yO E Y=Y,, constraint r is wredundant. Case 2aii. From (4.11) and (4.9), we have E, > 0, X, < 0, and hold. Case 2b. X, < 0, X2 < 0. Let y E Y. Then we obtain X, X, Also, by (4.5), S' = {y E R21 = {y E R21 Therefore, since y E S', X, implies that X, arbitrary, it follows that Y c H', which contradicts Claim 3. So, Case 2b cannot hold. Case 2c. X, > 0, X2 > 0. Let E S'nS2. Then X2 > 0, we obtain X, X\ that Y, = Y, so constraint r is redundant in (4.1) by Definition 4.1.3. Since, in addition, E Y = Y,, constraint r is wredundant. Since all possible cases for X, and X2 have been examined, it follows that constraint r is wredundant. Recall that the choice of r was arbitrary among the constraints 3,..,t. Hence for any i, i E {3,..,t}, constraint i is wredundant. Now, construct the tableau T,. for yo that corresponds to the basis B,.. Then in T,., yi, Y2, s3,...,s, are the basic variables. Also, there are (kt) slack variables different from s3,...,s, that are basic, but at a positive level, and s, and s2 are nonbasic. Let r E {3,...,t}. Suppose that s, is a basic variable for the q' row of the tableau. Consider this row of the tableau as the objective function for the linear programming problem min s,(y), subject to y E Y,. Since s, = 0, T,. is an optimal tableau for this problem (see Definition 4.1.3 and 4.1.4). Thus the optimality condition holds, and at,, 0 for j= 1,2 in T,., where aoi, j = 1,2, are the coefficients in the q"h row of T,. that correspond to the nonbasic variables s, and s2. Since r E {3,..,t} was arbitrary, for any s, that is basic and equal to zero in T,., say, for the q" row, au < 0, j = 1,2. We also know that in each of the columns for the 91 nonbasic variables s, and s2, there exists at least one positive coefficient. Since this positive coefficient cannot be in one of the rows with zero righthandside value, it must be in a row with a positive righthandside value. Hence, pivoting on this element of the tableau yields a neighboring extreme point of yo. Therefore, the edges E, and E, can be obtained in this way, and this completes the proof. Theorem 4.2.2. Assume that p = 2 and that d is linearly dependent on the rows of C. Then there exists an optimal solution x* for problem (PD) which belongs to Xx and which is also an optimal solution to at least one of the linear programs (LPu8), (LP,), and (LP2), where, for each i = 1,2, problem (LP) is given by (LP.) max Proof. Let w E R2 satisfy dT = wTC. Since p = 2, Y is a nonempty, compact polyhedron in R2. Consider the problem (PY) given by (PY) max Since Y is compact and the feasible region of problem (PY) is YE, based on a result of Shachtman (1974), we may assume without loss of generality that for some k >1, Y = {y E R2IMy < u, y 0}, where M is a kx2 matrix and u E R'. By Theorem 3.1.1, we may choose an optimal solution y* for problem (PY) which belongs to Yx. To prove the theorem, we will first show that y* maximizes at least one of y,, y2 and < w,y > over Y. There are four cases to consider. Case 1. y, = y; = 0. Then, since Y c {y E R2 y y 0} and y' E YE, Y = {0}. Therefore, in this case, y* maximizes both y, and yz over Y. Case 2. y' > 0 and y; = 0. Suppose y E Y. Then, since Y c {y E R2 y 0}, y2 y; = 0. Since y* E YE, this implies that y, < y,. Therefore, y* maximizes y, over Y. Case 3. y = 0 and y > 0. Then by a similar argument to the one given for Case 2, y* maximizes Y2 over Y. Case 4. y > 0 and y > 0. Let I, denote the kxk identity matrix. From linear programming theory, since y* is an extreme point of Y, we may choose a basic feasible solution for the system My' + Iy2 = u, y',y2 > 0, which corresponds to y' = y*. After some possible column rearrangements, this basic feasible solution can be represented by the tableau T given by yN yB D O g A Ik b where yN E R2 is the vector of nonbasic variables, and yB E Rk is the vector of basic variables in the basic feasible solution, and y* = g + DyN = g (see Ecker and Kouada 