Australian Research Council - Australian Postdoctoral Fellowship -  Project Period: 1997-1999
Title: Multicriteria methodology in operations research
Abstract: The concept of multicriteria consideration in operations research has been around for several decades. This has its origin in economics, but has also found its way into many other areas such as control theory, finance and engineering.  While the title of this proposal refers only to Operations Research,  it is really meant for all of the above mentioned areas as well.

Hitherto, most multicriteria models are formulated in terms of multicriteria, or vector, optimization problems.  This is because optimization problems have a clear physical interpretation and can be easily understood by most operations research practitioners.  There are numerous problems arising in operations research,  finance, control theory, and economics that cannot be adequately accounted for by a mere vector optimization formulation.  As such more generalized models have been  invented to overcome this limitation. These are: vector variational inequality, order complementarity and vector optimal control problems.  These extension models usually demand more sophisticated mathematics than those familiar to most operations research practitioners,  and as such have received much less attention than what their usefulness warrants.

In this proposed project, vector optimization problems (VO), vector variational inequality problems (VVI),  order complementarity problems (OC), vector optimal control  problems (VOC), and other related multicriteria  extensions will be studied in detail. In particular, we will

1. investigate their optimality properties and relationships with one another;

2. develop effective algorithms for solving these problems  and implement them on computers;

3. apply the theory and algorithms  to several problems arising from operations research, finance and control theory (see the next section for a detail discussion of these);

4. investigate equivalent VO, VVI, OC or VOC formulations for other known operations research models in the hope of a more effective solution to some
otherwise intractable or difficult problems; and

5. search for potential applications in relevant industries such as the Department of main roads and Telecom.

Research Committee of The Hong Kong Polytechnic University
Departmental small grants (2)
Project Period: 2000
Title: Nonconvex and Discrete Optimization in Operations Research
Abstract: Optimization is concerned with making decisions to balance various conflicting objectives via optimally distributing scarce resources. In particular, convex and continuous optimisation theory has found tremendous applications in a wide range of operations research problems. However many realistic problems are not blessed with these structures due to factors such as the complex nature of the problem and the integral nature of the decision variables. In spite of the well-known fact that nonconvex and discrete models are extremely useful in practice, there have been relatively few solution methods for solving these problems.

This project seeks to study characterisations of both global and local optimal solutions for optimisation problems with max-min type functions, to investigate numerical methods and algorithms for solving these optimization problems by virtue of nonlinear Lagrangian methods, to implement them on computers and to identify classes of nonconvex (network) optimization problems that can be convexified in terms of nonlinear Lagrangian functions. Our expected results should provide new methodologies for solving nonconvex and/or discrete optimisation problems.

Project Period: 2001
Title: Semi-infinite and Semi-definite Programming: Optimality Duality and Discretization
Abstract: The aim of this project is to study semi-infinite and semi-definite programming (SIDP) problems in a unified way. Several applications of semi-definite programming arise in statistics, pattern separation by ellipsoids and control theory. The semi-definite programming model includes linear programming, non-convex programming and combinatorial optimization problems as special cases. Semi-infinite programming is another class of important problems which has applications in filter design, circuit design and control system design.

We will explore first and second order necessary and sufficient optimality conditions for nonlinear SIDP problems. We will also study properties of generalized representation conditions and establish second order global sufficient optimality conditions for SIDP. We plan to study the duality theory of linear SIDP problems and to investigate the notion of generalized convexity for matrix-valued mappings and establish duality theory for nonlinear SIDP under generalized convexity. We will investigate a discretization method for solving linear SIDP problems, establish convergence results and carry out numerical testing.

Inter-faculty grant (1)
Project Period: 2003
Title: Supply Chain Modeling and Network Optimization
Abstract: This research applies and develops the most updated results in multiple objective programming and network analysis to study the supply chain modeling, in particular, to investigate the construction of supply network and the management in related industries crossing Hong Kong and the Chinese mainland. Different and conflicting objectives are considered simultaneously. The stochastic characteristics of the system are modeled to reflect the quickly updated, and uncertain information. We construct a general framework for supply chain design of multiple objective and stochastic nature based on the network equilibrium technology. With the general framework, we study  problems of location, transportation and logistics, inventory, and the profit management in supply chain management. The project is of both academic and practical significance. It is expected that the research reveal the potential value and opportunities for better theoretical improvements as well as practical applications.
 

RGC Earmarked Grant of the Research Grants Council of Hong Kong (6)
Project Period: 2000-2002
Title: Non-convex Multiple Criteria Optimization
Abstract: Optimization is concerned with making decisions to balance various conflicting objectives via optimally distributing scarce resources. There have been many successful applications of multiple criteria optimization in business and engineering, such as locating industrial plants, planning capital investments, and transporting goods. The global optimality condition, however, can be only achieved for multiple criteria optimization when certain convexity is present. The reality is that many multiple criteria decision-making problems are non-convex, due to factors such as the fixed charge or the integral nature of the decision variables. In such general non-convex situations, the traditional multiple criteria optimization methodologies do not produce a proper relationship between the set of the minimum points of the primal problem and the set of the maximum points of its corresponding Lagrangian dual problem, thus resulting a duality gap. Advancement in non-convex multiple criteria optimization will have a significant impact on improving the current practice of design, operation, and management in many fields of business and engineering.

This project seeks to establish a general theoretical framework for non-convex multiple criteria optimization by virtue of a well known fact that there always exists a supporting cone to the epi-graph of the perturbation function of non-convex multiple criteria optimization problems. The overall research goal will be accomplished by addressing the following research subjects: nonlinear Lagrangian duality theory, convergence analysis and exact penalization of nonlinear penalty functions, and optimality conditions of multiple criteria optimization problems in the absence of convexity. The developed theory will be applied to vector variational inequality problems and vector equilibrium problems. Our anticipated results should make a marked progress in advancing both the theory and practice of the state-of-the-art in non-convex multiple criteria optimization.

Project Period: 2001-2004
Title: Nonlinear Penalty Method for Constrained Global Optimization
Abstract: The penalty function method has been one of the most popular and efficient methods for solving constrained global optimization problems. The theoretical basis for the successful application of this method is the existence of an exact penalty function and the convergence of an optimal path generated by penalty functions to the optimal solution set. However, there are few well known difficulties associated with the conventional penalty type approach: rather slow convergence, and the Hessian of the penalty function is ill-conditioned. The research effort in recent years has been concentrated on generalizing the family of penalty function methods for overcoming these difficulties.

It is the objective of this project to set up a theoretical framework of nonlinear penalty function methods for solving constrained global optimization problems. We will establish verifiable sufficient conditions for the existence of exact nonlinear penalty functions by exploring some second order sufficient conditions. We will seek conditions that guarantee the existence and convergence of an optimal path generated by nonlinear penalty functions for nonlinear and nonconvex optimization problems. We will apply the established results of nonlinear penalty functions to mathematical programs with vector variational inequality constraints. Our anticipated results on exact nonlinear penalty functions and convergence analysis should make a significant progress in advancing the state-of-art in constrained global optimization, which is of significance to the overall optimal decision making tasks in engineering and decision support areas.

Project Period: 2002-2004
Title: Estimating of Parameters in Penalty Type Methods
Abstract: Classical Lagrange function does not work well for many non-convex optimization problems. Application of penalization does not lead to good results for some non-convex problem. One important issue for the numerical implementation of penalty-type approaches is that a fairly small exact penalty parameter is required. However sometimes the exact penalty parameter is very large, even though it exists, which is an obstacle for numerical implementation of penalty methods. In particular, classical penalty function has no advantage when applying to concave programming problems because of a large penalty parameter. Thus the question arises how to generalize classical Lagrange and penalty functions, in order to obtain an appropriate scheme for reducing constrained optimization problems to unconstrained optimization problems for sufficiently large classes of optimization problems.

In this project we will investigate nonlinear analogues of Lagrange and penalty functions: a generalized augmented Lagrangian scheme with a level-bounded augmenting function, which may not necessarily be convex and a nonlinear penalty scheme, which results in diminishing of the least exact penalty parameter. We aim to obtain estimates of parameters, which appear in our approach, and estimates of the least exact penalty parameter for several classes of problems, and to establish a theoretical foundation of a unified approach to optimality and duality via a monotone composite optimization model. We will apply the anticipated results to the portfolio selection problem subject to transaction cost and Value-at-Risk constraint respectively.  The outcome of this project will include a deeper understanding of the role of Lagrange multipliers in generalized augmented Lagrangian method and penalty parameters in penalty function method, and new practical numerical optimization methods.

Project Period: 2003-2005
Title: Optimization Methods with Lower Order Merit Functions
Abstract: Recently several classes of lower order merit functions, such as nonconvex nonsmooth penalty functions, nonlinear penalty functions and generalized augmented Lagrangian functions, have received intensive attention. These lower order merit functions are in general not Lipschitz. It has been established that lower order merit functions have some theoretical advantages: existence of exact penalty parameters for constrained optimization problems under generalized analytic properties, which are weaker than the regularity condition required for the classical penalty function, and possession of a smaller least exact penalty parameter comparing to that of the classical penalty function.

In this project, we will study sequential quadratic programming methods with lower order merit functions and establish global and superlinear convergence properties. We will investigate smoothing approximation methods for solving penalty problems with lower order merit functions and necessary and sufficient optimality conditions of these penalty problems. We will apply the designed methods to mathematical programs with complementarity constraints and the implied volatility surface problem of American options. The outcome of this project will include a deeper understanding of optimization methods with lower order merit functions, their convergence properties and optimality conditions.

Project Period: 2004-2007
Title: Infinite Dimensional Optimization with Applications
Abstract: It is well known that infinite dimensional optimization problems have already found many applications in engineering, game theory, optimal control and stochastic programming. More applications of infinite dimensional optimization have recently been found in the American option pricing and the continuous and time-dependent vector traffic equilibrium principle. In particular, the application of a lower order power penalty function to the linear complementarity problem arising from the American option pricing provides a higher order convergence rate, which is better than the square root convergence rate of the linear penalty function in the sense that a smaller penalty parameter is only required to achieve the same accuracy. Our early study on the generalized augmented Lagrangian approach in finite dimensional spaces has established a general zero duality gap property and a smaller exact penalty parameter.

This project aims to study infinite dimensional optimization theory and methods in the framework of generalized augmented Lagrangian functions. We will establish an exact penalization representation; and explore convergence analysis and convergence rates of a multiplier-type method for an equality constrained optimization problem. As applications, we will establish the existence of a finite penalty parameter such that a solution of the penalized Black-Scholes partial differential equation (PDE) is that of the original PDE with the obstacle constraint in the American option pricing and investigate mathematical programs with vector variational inequality constraints by virtue of a gap function approach. The anticipated results will be a deeper understanding of the way optimization problems in infinite dimension can be solved, which lead to applications of some Operations Research and Mathematical Finance problems.

Project Period: 2005-2008
Title: Semi-Definite Programs with Applications
Abstract: Semi-definite programs (SDP for short) involve the minimization of an objective function subject to positive semi-definite matrix constraints. Extensive study of SDP has been driven by the facts that SDP has important applications in statistics, finance, and structural design and that SDP approach offers a unified way to study theories and algorithms for a wide variety of convex and non-convex optimization problems. It has been shown that generalized augmented Lagrangian and nonlinear Lagrangian schemes have the following properties for nonlinear optimization: a general zero duality gap property without any convexity assumption, necessary and sufficient condition of the existence of an exact penalty parameter and a diminishable least exact penalty parameter.

In this project, we will investigate duality, exact penalty and optimality of nonlinear SDP via generalized augmented Lagrangian. We will explore equivalence between zero duality gap properties in terms of augmented Lagrangian function and nonlinear Lagrangian function respectively. This equivalence will be useful in better understanding characterizations of different duality schemes. We will explore reformulation of a semi-infinite linear SDP such that the corresponding Lagrangian dual scheme provides a zero duality gap without any constraint qualification. We will apply the established results to various financial optimization problems.

Project Period: 2007-2008
Title: Second-order Optimality Conditions in Constrained Optimization

Project Period: 2008-2009
Title: Vector Variational Inequalities with Applications

Project Period: 2009-2011
Title: Semi-infinite and Generalized Semi-infinite Programs

Project Period: 2012-2014
Title: Generalized Polyhedral Theory with Applications

Project Period: 2013-2015
Title: Generalized Penalty Functions and Methods

Project Period: 2014-2016
Title: A Unified Nonlinear Augmented Lagrangian Theory

Project Period: 2016-2018
Title: Nonlinear Constrained Sparse Optimization with Lower Order Regularization

Project Period: 2017-2019
Title: Linearized Proximal Algorithms with Bregman Distance for Convex Composite Optimization with Applications, (Co-I: Dr Burachik Regina)

Project Period: 2018-2020
Title: Variational Analysis of Piecewise Linear Vector Optimization

Project Period: 2019-2021
Title: Error Bounds and their Stability Analysis and Applications
Title: Second-order Optimality Conditions in Constrained Optimization

Project Period: 2007-2008
Title: Vector Variational Inequalities with Applications

Project Period: 2008-2009

Title: Semi-infinite and Generalized Semi-infinite Programs
Project Period: 2009-2011

Title: Generalized Polyhedral Theory with Applications
Project Period: 2012-2014

Title: Generalized Penalty Functions and Methods
Project Period: 2013-2015

Title: A Unified Nonlinear Augmented Lagrangian Theory
Project Period: 2014-2016

Title: Nonlinear Constrained Sparse Optimization with Lower Order Regularization (RGC Ref No.15216715)
Project Period: 1.2016-12.2018
Abstract: The sparse solutions of under-determined nonlinear measurements arise in the sparse inverse covariance selection while maximizing the log-likelihood, quadratic compressive sensing in diffraction imaging and sub-wavelength imaging, and nonlinear base pursuit. Nearly, all of the optimization theory and algorithms developed for compressed sensing signal recovery assume that samples are taken using linear measurements and without constraints. Thus it is of great demand to design optimal algorithms to solve constrained sparse optimization problems with nonlinear measurements.
In this project, we will address compressed sensing recovery problems in a setting where the observations are nonlinear measurements and there are constraints and develop optimization theory and algorithms by virtue of constrained lower order regularization problems for solving them. We will introduce the extended restricted isometry property and extended restricted eigenvalue condition, and apply them to establish the recovery bound for the lower order regularization problem of the constrained sparse optimization with nonlinear measurements. We will propose a projected proximal gradient method to solve the constrained lower order regularization problem, and establish the convergence theory by virtue of the projection operator and the Kurdyka-Lojasewicz theory. As optimization problems are of a nonconvex structure, it is difficult to establish that our algorithms converge to a global solution. However, we will aim to design algorithms to find a good enough sparse solution and compare their performance with some existing optimization algorithms in terms of computing time, accuracy and successful recovery rate.
We will also apply our theory and algorithms to solve (group) sparse constrained optimization problems arising in portfolio selection and gene regulatory network. The practical meaning of the sparcity requirement in the portfolio is that an investor aims to invest in a small number of assets in order to avoid the high cost of transaction fees. The group sparcity structure arises in prediction, dynamic MRI, and gene finding. We will show that exploiting the group sparcity structure can reduce the degrees of freedom in the solution, thereby lead to better recovery performance.

Title: Linearized Proximal Algorithms with Bregman Distance for Convex Composite Optimization with Applications (Co-I: Dr Burachik Regina) (RGC Ref No. 15234216)
Project Period: 1.2017-12.2019
Abstract: The proximal point algorithm has been very popular in solving structured optimization problems for more than half a century. One of the main reasons is that they generate subproblems that always have a unique solution, and are able to avoid the ill conditioning in numerical implementation. A well-known drawback of the proximal point algorithms is that the computational cost of solving each subproblem is as high as solving the original problem. We will employ a linearization technique and a Bregman distance to overcome this drawback, such that the corresponding subproblem is convex, irrespective of the original problem being convex or not, and remains to have a unique solution which lies in the interior of the feasible set so that the constraints are thus eliminated automatically. In recent years, there has been an extensive study on the global convergence of proximal point algorithms for convex constrained optimization problems. However, there is a lack of research on the convergence rates of proximaltype algorithms with Bregman distance for both convex and nonconvex constrained optimization problems. It is well known that the establishment of convergence rates is important in guaranteeing the numerical performance of relevant algorithms.
In this project, we will develop a linearized proximal theory with Bergman distance for solving convex composite optimization, including local superlinear convergence rates in terms of both functional values and iterates, under the assumptions of local weak sharp minima of order p and a quasi-regularity condition. We will propose a globalization strategy for the linearized proximal algorithm with Bregman distance based on the Armijo-line search and establish global superlinear convergence results. We will apply the designed linearized proximal algorithms with Bergman distance to a feasibility problem and a sensor network localization problem. The project is of a long-term significant value, with an implication of further investigation on effective solution algorithms for constrained optimization problems and in the training of Operations Researchers with strong mathematical skills.

Title: Variational Analysis of Piecewise Linear Vector Optimization (RGC Ref No. 15212817)
Project Period: 1.2018-12.2020
Abstract: Linear and continuous piecewise linear vector optimization have been extensively studied and applied to various decision-making problems in economics, management science and engineering. The construction theory of solution sets of vector optimization problem in the case of finite dimension spaces with the polyhedron ordering cone has played a key role in the development of its theory and applications. It is well recognized that continuous piecewise linearity is still very restrictive in both theory and application. Practical problems of piecewise linear functions include transaction costs in finance and sparse nature of the decision variables in statistics and compress sensing. Here by a piecewise linear function we mean a not necessarily continuous one. It is worth noting that the study of piecewise linear, in particular vector, optimization problems is very limited and immature. Therefore there is a need to establish a unified and in-depth theory for piecewise linear vector optimization problems.
In this project, we will study the fully piecewise linear vector optimization problem, that is, both objective and constraint functions can be piecewise linear. We will first explore the solution structures of the piecewise linear vector optimization problem in general Banach spaces and apply them to develop weak sharp minima for linear vector optimization problems and to explore the corresponding finite terminate rule for some piecewise linear vector optimization algorithms. On the other hand, in establishing mathematical models of practical problems, one often adopts piecewise linear functions as objectives and constraint functions, and these piecewise linear functions are determined by only finitely many available test data. However all test data obtained are not perfectly ideal. In general, the gap between the ideal data with the ``test" data is unavoidable. In view of this, we will consider some stability issues when the objective and/or constraint functions undergo small perturbations.

Title: Error Bounds and their Stability Analysis and Applications (RGC Ref No. 15216518)
Project Period: 1.2019-12.2021
Abstract: Error bounds and related topics, such as metric subregularity, weak sharp minima, calmness and exact penalization, have been extensively studied in last three decades. Error bounds have important applications in the design of optimal algorithms and the study of their convergence analysis in solving nonlinear optimization problems, as convergence rates of some iterative algorithms rely on the error bound property and the accuracy of the approximate solution obtained by the algorithms depends on the magnitude of the local error bound modulus. As in the practical optimization applications, data in the models are often not estimated accurately, stability analysis can guarantee that the solutions or properties of the original system can be reserved under certain kinds of perturbations. However the study on the computable local error bound moduli and their stability analysis is relatively new and immature. Therefore there is a need to carry out an in-depth study for local error bound moduli and their stability analysis and applications.
In this project we will establish computable lower and upper estimates of the local error bound modulus for a proper and lower semicontinuous function without regularity assumptions and obtain a computable global error bound modulus for a positively homogeneous function by virtue of variational analysis tools. We will provide formulae for the uniform local error bound modulus and the restricted uniform local error bound modulus respectively for a proper and lower semicontinuous function. We will exam sufficient conditions for uniform local error bound and the restricted uniform local error bound properties that allow us to explore stability analysis of the relevant error bound. We will apply the obtained results to the study of the convergence analysis of the sequence produced by approximate problems of a difference-of-convex optimization problem to a Bouligand-stationary point and investigate the local error bound property of the supremum function of a linear portfolio selection optimization problem with a sparsity constraint.

Title: Stability Analysis of Generalised Equations with Applications (RGC Ref No. 15218219)
Project Period: 1.2020-12.2022
Abstract: Generalized equations in optimization include linear and nonlinear inequality systems and linear and nonlinear variational inequality problems. In practical application problems, basic data in the models such as demand, prices, resources, technical parameters etc. are often not estimated accurately. Stability analysis of generalized equations is to determine intuitively verifiable conditions to guarantee that the accuracy of the solutions obtained increase with the degree of approximation of the initial data. The literature on the subject is vast. However existing results of generalized equations are mainly concentrated on canonical perturbation of linear and nonlinear inequality systems and linear and nonlinear variational inequality problems.
This project will study stability of generalized equations with an explicit set constraint and the matrix perturbation of a linear constraint system. These considerations are motivated by the facts that the explicit treatment of the set constraint allows one to use its special structure when calculating its normal cone and that the projection on the intersection of two simple polyhedral sets may be easily computed by alternating projection onto each simple polyhedral set. Moreover it is worth noting that the matrix perturbation of a linear constraint system arises in some linear portfolio selection optimization problems. Our approach relies on the Mordukhovich criterion, a remarkable variational analysis tool in the study of stability. We will study the Lipschitz-like continuity of the set of solutions of a fully perturbed linear constraint system with a set constraint and apply the obtained results to sparse linear complementarity problems. We will investigate the Lipschitz-like property of the stationary set of linear and nonlinear variational inequality problems with a matrix perturbation in the linear constraint system by virtue of a unconstrained minimization approach and a set constraint approach respectively. We will apply the obtained results to linear portfolio optimization problems.

Title: The Lipschitz-like Property Relative to a Set with Applications (RGC Ref No. 15217520)
Project Period: 1.2021-12.2023
Abstract: Stability analysis of set-valued mappings is to determine intuitively verifiable conditions to guarantee that the accuracy of the solutions obtained increases with the degree of approximation of the initial data. The literature on the subject is vast when the study is of global nature. Lipschitz-like property is one of the properties at the center of stability theory. By virtue of coderivative, Mordukhovich criterion provides a necessary and sufficient condition of the Lipschitz-like property and has widely been applied in the study of solution mappings of various linear and nonlinear systems. One important assumption of the Mordukhovich criterion is that the candidate point is in the interior of the domain of set-valued mappings. This is a very restrictive assumption. On one hand, the candidate point under consideration may be at the boundary of the domain of setvalued mappings. On the other hand, in order to guarantee some stationarity conditions one may only need a regular behavior of the constraint systems with respect to one single critical direction, not on the whole space. Thus there is a great demand to study characterizations of Lipschitz-like property relative to a set.
This project will investigate characterizations of the Lipschitz-like property relative to a set for set-valued mappings and their applications. We will employ regular normal cone and limiting normal cone of a restricted graph of the set-valued mapping to obtain some neighborhood necessary condition and sufficient condition respectively for set-valued mappings to have the Lipschitz-like property relative to a closed set. We will introduce a projectional coderivative of set-valued mappings and apply it to establish a verifiable generalized Mordukhovich criterion for the Lipschitz-like property relative to a closed and convex set. We will study the representation of the graphical modulus of a setvalued mapping relative to a closed and convex set by using the outer norm of the corresponding projectional coderivative value. We will apply the obtained results to investigate the Lipschitz-like property relative to a set for a level-set mapping and the stationary point set of an l_1-norm convex optimization problem.

Back home