Generalised rational approximation and its application to improve deep learning classifiers
- Authors: Peiris, V , Sharon, Nir , Sukhorukova, Nadezda , Ugon, Julien
- Date: 2021
- Type: Text , Journal article
- Relation: Applied Mathematics and Computation Vol. 389, no. (2021), p.
- Relation: https://purl.org/au-research/grants/arc/DP180100602
- Full Text: false
- Reviewed:
- Description: A rational approximation (that is, approximation by a ratio of two polynomials) is a flexible alternative to polynomial approximation. In particular, rational functions exhibit accurate estimations to nonsmooth and non-Lipschitz functions, where polynomial approximations are not efficient. We prove that the optimisation problems appearing in the best uniform rational approximation and its generalisation to a ratio of linear combinations of basis functions are quasiconvex even when the basis functions are not restricted to monomials. Then we show how this fact can be used in the development of computational methods. This paper presents a theoretical study of the arising optimisation problems and provides results of several numerical experiments. We apply our approximation as a preprocessing step to deep learning classifiers and demonstrate that the classification accuracy is significantly improved compared to the classification of the raw signals. © 2020
- Description: This research was supported by the Australian Research Council (ARC), Solving hard Chebyshev approximation problems through nonsmooth analysis (Discovery Project DP180100602 ). This research was partially sponsored by Tel Aviv-Swinburne Research Collaboration Grant (2019).
Lipschitz modulus of linear and convex inequality systems with the Hausdorff metric
- Authors: Beer, Gerald , Cánovas, M. J. , López, Marco , Parra, Juan
- Date: 2021
- Type: Text , Journal article
- Relation: Mathematical Programming Vol. 189, no. 1-2 (2021), p. 75-98. https://purl.org/au-research/grants/arc/DP180100602
- Relation: http://purl.org/au-research/grants/arc/DP180100602
- Full Text:
- Reviewed:
- Description: This paper analyzes the Lipschitz behavior of the feasible set mapping associated with linear and convex inequality systems in Rn. To start with, we deal with the parameter space of linear (finite/semi-infinite) systems identified with the corresponding sets of coefficient vectors, which are assumed to be closed subsets of Rn+1. In this framework the size of perturbations is measured by means of the (extended) Hausdorff distance. A direct antecedent, extensively studied in the literature, comes from considering the parameter space of all linear systems with a fixed index set, T, where the Chebyshev (extended) distance is used to measure perturbations. In the present work we propose an appropriate indexation strategy which allows us to establish the equality of the Lipschitz moduli of the feasible set mappings in both parametric contexts, as well as to benefit from existing results in the Chebyshev setting for transferring them to the Hausdorff one. In a second stage, the possibility of perturbing directly the set of coefficient vectors of a linear system leads to new contributions on the Lipschitz behavior of convex systems via linearization techniques. © 2020, Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society. Correction: The article “Lipschitz modulus of linear and convex inequality systems with the Hausdorff metric”, written by Beer,G., Cánovas, M.J., López, M.A., Parra, J.was originally published Online First without Open Access. After publication in volume 189, issue 1–2, page 75–98 the author decided to opt for Open Choice and to make the article an Open Access publication. Therefore, the copyright of the article has been changed to © The Author(s) 2020 and the article is forthwith distributed under the terms of the Creative Commons Attribution 4.0 International License. https://doi.org/10.1007/s10107-021-01751-x
- Description: This paper analyzes the Lipschitz behavior of the feasible set mapping associated with linear and convex inequality systems in Rn. To start with, we deal with the parameter space of linear (finite/semi-infinite) systems identified with the corresponding sets of coefficient vectors, which are assumed to be closed subsets of Rn+1. In this framework the size of perturbations is measured by means of the (extended) Hausdorff distance. A direct antecedent, extensively studied in the literature, comes from considering the parameter space of all linear systems with a fixed index set, T, where the Chebyshev (extended) distance is used to measure perturbations. In the present work we propose an appropriate indexation strategy which allows us to establish the equality of the Lipschitz moduli of the feasible set mappings in both parametric contexts, as well as to benefit from existing results in the Chebyshev setting for transferring them to the Hausdorff one. In a second stage, the possibility of perturbing directly the set of coefficient vectors of a linear system leads to new contributions on the Lipschitz behavior of convex systems via linearization techniques. © 2020, Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society.
On computation of optimal strategies in oligopolistic markets respecting the cost of change
- Authors: Outrata, Jiri , Valdman, Jan
- Date: 2020
- Type: Text , Journal article
- Relation: Mathematical Methods of Operations Research Vol. 92, no. 3 (2020), p. 489-509
- Relation: http://purl.org/au-research/grants/arc/DP160100854
- Full Text:
- Reviewed:
- Description: The paper deals with a class of parameterized equilibrium problems, where the objectives of the players do possess nonsmooth terms. The respective Nash equilibria can be characterized via a parameter-dependent variational inequality of the second kind, whose Lipschitzian stability, under appropriate conditions, is established. This theory is then applied to evolution of an oligopolistic market in which the firms adapt their production strategies to changing input costs, while each change of the production is associated with some “costs of change”. We examine both the Cournot-Nash equilibria as well as the two-level case, when one firm decides to take over the role of the Leader (Stackelberg equilibrium). The impact of costs of change is illustrated by academic examples. © 2020, Springer-Verlag GmbH Germany, part of Springer Nature.
Polytopes close to being simple
- Authors: Pineda-Villavicencio, Guillermo , Ugon, Julien , Yost, David
- Date: 2020
- Type: Text , Journal article
- Relation: Discrete and Computational Geometry Vol. 64, no. 1 (2020), p. 200-215
- Relation: http://purl.org/au-research/grants/arc/DP180100602
- Full Text:
- Reviewed:
- Description: It is known that polytopes with at most two nonsimple vertices are reconstructible from their graphs, and that d-polytopes with at most d- 2 nonsimple vertices are reconstructible from their 2-skeletons. Here we close the gap between 2 and d- 2 , showing that certain polytopes with more than two nonsimple vertices are reconstructible from their graphs. In particular, we prove that reconstructibility from graphs also holds for d-polytopes with d+ k vertices and at most d- k+ 3 nonsimple vertices, provided k
A sharp augmented Lagrangian-based method in constrained non-convex optimization
- Authors: Bagirov, Adil , Ozturk, Gurkan , Kasimbeyli, Refail
- Date: 2019
- Type: Text , Journal article
- Relation: Optimization Methods and Software Vol. 34, no. 3 (2019), p. 462-488
- Full Text: false
- Reviewed:
- Description: In this paper, a novel sharp Augmented Lagrangian-based global optimization method is developed for solving constrained non-convex optimization problems. The algorithm consists of outer and inner loops. At each inner iteration, the discrete gradient method is applied to minimize the sharp augmented Lagrangian function. Depending on the solution found the algorithm stops or updates the dual variables in the inner loop, or updates the upper or lower bounds by going to the outer loop. The convergence results for the proposed method are presented. The performance of the method is demonstrated using a wide range of nonlinear smooth and non-smooth constrained optimization test problems from the literature.
On the reconstruction of polytopes
- Authors: Doolittle, Joseph , Nevo, Eran , Pineda-Villavicencio, Guillermo , Ugon, Julien , Yost, David
- Date: 2019
- Type: Text , Journal article
- Relation: Discrete and Computational Geometry Vol. 61, no. 2 (2019), p. 285-302. http://purl.org/au-research/grants/arc/DP180100602
- Full Text:
- Reviewed:
- Description: Blind and Mani, and later Kalai, showed that the face lattice of a simple polytope is determined by its graph, namely its 1-skeleton. Call a vertex of a d-polytope nonsimple if the number of edges incident to it is more than d. We show that (1) the face lattice of any d-polytope with at most two nonsimple vertices is determined by its 1-skeleton; (2) the face lattice of any d-polytope with at most d- 2 nonsimple vertices is determined by its 2-skeleton; and (3) for any d> 3 there are two d-polytopes with d- 1 nonsimple vertices, isomorphic (d- 3) -skeleta and nonisomorphic face lattices. In particular, the result (1) is best possible for 4-polytopes. © 2018, Springer Science+Business Media, LLC, part of Springer Nature.
Two curve Chebyshev approximation and its application to signal clustering
- Authors: Sukhorukova, Nadezda
- Date: 2019
- Type: Text , Journal article
- Relation: Applied Mathematics and Computation Vol. 356, no. (2019), p. 42-49
- Relation: http://purl.org/au-research/grants/arc/DP180100602
- Full Text:
- Reviewed:
- Description: In this paper, we extend a number of important results of the classical Chebyshev approximation theory to the case of simultaneous approximation of two or more functions. The need for this extension is application driven, since such kind of problems appears in the area of curve (signal) clustering. In this paper, we propose a new efficient algorithm for signal clustering and develop a procedure that allows one to reuse the results obtained at the previous iteration without recomputing the cluster centres from scratch. This approach is based on the extension of the classical de la Vallee-Poussin procedure originally developed for polynomial approximation. We also develop necessary and sufficient optimality conditions for two curve Chebyshev approximation, which is our core tool for curve clustering. These results are based on application of nonsmooth convex analysis. (C) 2019 Elsevier Inc. All rights reserved. In this paper, we extend a number of important results of the classical Chebyshev approximation theory to the case of simultaneous approximation of two or more functions. The need for this extension is application driven, since such kind of problems appears in the area of curve (signal) clustering. In this paper, we propose a new efficient algorithm for signal clustering and develop a procedure that allows one to reuse the results obtained at the previous iteration without recomputing the cluster centres from scratch. This approach is based on the extension of the classical de la Vallee-Poussin procedure originally developed for polynomial approximation. We also develop necessary and sufficient optimality conditions for two curve Chebyshev approximation, which is our core tool for curve clustering. These results are based on application of nonsmooth convex analysis. (C) 2019 Elsevier Inc. All rights reserved.
Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations
- Authors: Gaudioso, Manlio , Giallombardo, Giovanni , Miglionico, Giovanna , Bagirov, Adil
- Date: 2018
- Type: Text , Journal article
- Relation: Journal of Global Optimization Vol. 71, no. 1 (2018), p. 37-55
- Full Text: false
- Reviewed:
- Description: We introduce a proximal bundle method for the numerical minimization of a nonsmooth difference-of-convex (DC) function. Exploiting some classic ideas coming from cutting-plane approaches for the convex case, we iteratively build two separate piecewise-affine approximations of the component functions, grouping the corresponding information in two separate bundles. In the bundle of the first component, only information related to points close to the current iterate are maintained, while the second bundle only refers to a global model of the corresponding component function. We combine the two convex piecewise-affine approximations, and generate a DC piecewise-affine model, which can also be seen as the pointwise maximum of several concave piecewise-affine functions. Such a nonconvex model is locally approximated by means of an auxiliary quadratic program, whose solution is used to certify approximate criticality or to generate a descent search-direction, along with a predicted reduction, that is next explored in a line-search setting. To improve the approximation properties at points that are far from the current iterate a supplementary quadratic program is also introduced to generate an alternative more promising search-direction. We discuss the main convergence issues of the line-search based proximal bundle method, and provide computational results on a set of academic benchmark test problems. © 2017, Springer Science+Business Media, LLC.
Nonsmooth DC programming approach to clusterwise linear regression : Optimality conditions and algorithms
- Authors: Bagirov, Adil , Ugon, Julien
- Date: 2018
- Type: Text , Journal article
- Relation: Optimization Methods and Software Vol. 33, no. 1 (2018), p. 194-219
- Relation: http://purl.org/au-research/grants/arc/DP140103213
- Full Text: false
- Reviewed:
- Description: The clusterwise linear regression problem is formulated as a nonsmooth nonconvex optimization problem using the squared regression error function. The objective function in this problem is represented as a difference of convex functions. Optimality conditions are derived, and an algorithm is designed based on such a representation. An incremental approach is proposed to generate starting solutions. The algorithm is tested on small to large data sets. © 2017 Informa UK Limited, trading as Taylor & Francis Group.
Perturbation of error bounds
- Authors: Kruger, Alexander , López, Marco , Théra, Michel
- Date: 2018
- Type: Text , Journal article
- Relation: Mathematical Programming Vol. 168, no. 1-2 (2018), p. 533-554
- Relation: http://purl.org/au-research/grants/arc/DP160100854
- Full Text:
- Reviewed:
- Description: Our aim in the current article is to extend the developments in Kruger et al. (SIAM J Optim 20(6):3280–3296, 2010. doi:10.1137/100782206) and, more precisely, to characterize, in the Banach space setting, the stability of the local and global error bound property of inequalities determined by lower semicontinuous functions under data perturbations. We propose new concepts of (arbitrary, convex and linear) perturbations of the given function defining the system under consideration, which turn out to be a useful tool in our analysis. The characterizations of error bounds for families of perturbations can be interpreted as estimates of the ‘radius of error bounds’. The definitions and characterizations are illustrated by examples. © 2017, Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society.
Set regularities and feasibility problems
- Authors: Kruger, Alexander , Luke, Russell , Thao, Nguyen
- Date: 2018
- Type: Text , Journal article
- Relation: Mathematical Programming Vol. 168, no. 1-2 (2018), p. 279-311
- Relation: http://purl.org/au-research/grants/arc/DP160100854
- Full Text:
- Reviewed:
- Description: We synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems. Several new characterizations of regularities are presented which shed light on the relations between seemingly different ideas and point to possible necessary conditions for local linear convergence of fundamental algorithms
A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes
- Authors: Joki, Kaisa , Bagirov, Adil , Karmitsa, Napsu , Makela, Marko
- Date: 2017
- Type: Text , Journal article
- Relation: Journal of Global Optimization Vol. 68, no. 3 (2017), p. 501-535
- Relation: http://purl.org/au-research/grants/arc/DP140103213
- Full Text: false
- Reviewed:
- Description: In this paper, we develop a version of the bundle method to solve unconstrained difference of convex (DC) programming problems. It is assumed that a DC representation of the objective function is available. Our main idea is to utilize subgradients of both the first and second components in the DC representation. This subgradient information is gathered from some neighborhood of the current iteration point and it is used to build separately an approximation for each component in the DC representation. By combining these approximations we obtain a new nonconvex cutting plane model of the original objective function, which takes into account explicitly both the convex and the concave behavior of the objective function. We design the proximal bundle method for DC programming based on this new approach and prove the convergence of the method to an -critical point. The algorithm is tested using some academic test problems and the preliminary numerical results have shown the good performance of the new bundle method. An interesting fact is that the new algorithm finds nearly always the global solution in our test problems.
On the Aubin property of a class of parameterized variational systems
- Authors: Gfrerer, Helmut , Outrata, Jiri
- Date: 2017
- Type: Text , Journal article
- Relation: Mathematical Methods of Operations Research Vol. 86, no. 3 (2017), p. 443-467
- Relation: http://purl.org/au-research/grants/arc/DP160100854
- Full Text:
- Reviewed:
- Description: The paper deals with a new sharp condition ensuring the Aubin property of solution maps to a class of parameterized variational systems. This class encompasses various types of parameterized variational inequalities/generalized equations with fairly general constraint sets. The new condition requires computation of directional limiting coderivatives of the normal-cone mapping for the so-called critical directions. The respective formulas have the form of a second-order chain rule and extend the available calculus of directional limiting objects. The suggested procedure is illustrated by means of examples. © 2017, Springer-Verlag GmbH Germany.
Global solutions to nonconvex optimization of 4th-order polynomial and log-sum-exp functions
- Authors: Chen, Yi , Gao, David
- Date: 2016
- Type: Text , Journal article
- Relation: Journal of Global Optimization Vol. 64, no. 3 (2016), p. 417-431
- Full Text:
- Reviewed:
- Description: This paper presents a canonical dual approach for solving a nonconvex global optimization problem governed by a sum of 4th-order polynomial and a log-sum-exp function. Such a problem arises extensively in engineering and sciences. Based on the canonical duality–triality theory, this nonconvex problem is transformed to an equivalent dual problem, which can be solved easily under certain conditions. We proved that both global minimizer and the biggest local extrema of the primal problem can be obtained analytically from the canonical dual solutions. As two special cases, a quartic polynomial minimization and a minimax problem are discussed. Existence conditions are derived, which can be used to classify easy and relative hard instances. Applications are illustrated by several nonconvex and nonsmooth examples. © 2014, Springer Science+Business Media New York.
Nonsmooth Lyapunov pairs for differential inclusions governed by operators with nonempty interior domain
- Authors: Adly, Samir , Hantoute, Abderrahim , Thera, Michel
- Date: 2016
- Type: Text , Journal article
- Relation: Mathematical Programming Vol. 157, no. 2 (2016), p. 349-374
- Full Text:
- Reviewed:
- Description: The general theory of Lyapunov stability of first-order differential inclusions in Hilbert spaces has been studied by the authors in the previous paper (Adly et al. in Nonlinear Anal 75(3): 985–1008, 2012). This new contribution focuses on the case when the interior of the domain of the maximally monotone operator governing the given differential inclusion is nonempty; this includes in a natural way the finite-dimensional case. The current setting leads to simplified, more explicit criteria and permits some flexibility in the choice of the generalized subdifferentials. Some consequences of the viability of closed sets are given. Our analysis makes use of standard tools from convex and variational analysis. © 2015, Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society.
On computation of generalized derivatives of the normal-cone mapping and their applications
- Authors: Gfrerer, Helmut , Outrata, Jiri
- Date: 2016
- Type: Text , Journal article
- Relation: Mathematics of Operations Research Vol. 41, no. 4 (2016), p. 1535-1556
- Full Text: false
- Reviewed:
- Description: The paper concerns the computation of the graphical derivative and the regular (Fréchet) coderivative of the normal-cone mapping related to C2 inequality constraints under very weak qualification conditions. This enables us to provide the graphical derivative and the regular coderivative of the solution map to a class of parameterized generalized equations with the constraint set of the investigated type. On the basis of these results, we finally obtain a characterization of the isolated calmness property of the mentioned solution map and derive strong stationarity conditions for an MPEC with control constraints. © 2016 INFORMS.
A heuristic algorithm for solving the minimum sum-of-squares clustering problems
- Authors: Ordin, Burak , Bagirov, Adil
- Date: 2015
- Type: Text , Journal article
- Relation: Journal of Global Optimization Vol. 61, no. 2 (2015), p. 341-361
- Relation: http://purl.org/au-research/grants/arc/DP140103213
- Full Text: false
- Reviewed:
- Description: Clustering is an important task in data mining. It can be formulated as a global optimization problem which is challenging for existing global optimization techniques even in medium size data sets. Various heuristics were developed to solve the clustering problem. The global k-means and modified global k-means are among most efficient heuristics for solving the minimum sum-of-squares clustering problem. However, these algorithms are not always accurate in finding global or near global solutions to the clustering problem. In this paper, we introduce a new algorithm to improve the accuracy of the modified global k-means algorithm in finding global solutions. We use an auxiliary cluster problem to generate a set of initial points and apply the k-means algorithm starting from these points to find the global solution to the clustering problems. Numerical results on 16 real-world data sets clearly demonstrate the superiority of the proposed algorithm over the global and modified global k-means algorithms in finding global solutions to clustering problems.
Directional metric regularity of multifunctions
- Authors: Ngai, Huynh Van , Thera, Michel
- Date: 2015
- Type: Text , Journal article
- Relation: Mathematics of Operations Research Vol. 40, no. 4 (2015), p. 969-991
- Relation: http://purl.org/au-research/grants/arc/DP110102011
- Full Text:
- Reviewed:
- Description: In this paper, we study relative metric regularity of set-valued mappings with emphasis on directional metric regularity. We establish characterizations of relative metric regularity without assuming the completeness of the image spaces, by using the relative lower semicontinuous envelopes of the distance functions to set-valued mappings. We then apply these characterizations to establish a coderivative type criterion for directional metric regularity as well as for the robustness of metric regularity.
- Description: In this paper, we study relative metric regularity of set-valued mappings with emphasis on directional metric regularity. We establish characterizations of relative metric regularity without assuming the completeness of the image spaces, by using the relative lower semicontinuous envelopes of the distance functions to set-valued mappings. We then apply these characterizations to establish a coderivative type criterion for directional metric regularity as well as for the robustness of metric regularity. © 2015 INFORMS.
Global optimality conditions and optimization methods for polynomial programming problems
- Authors: Wu, Zhiyou , Tian, Jing , Ugon, Julien
- Date: 2015
- Type: Text , Journal article
- Relation: Journal of Global Optimization Vol. 62, no. 4 (2015), p. 617-641
- Full Text: false
- Reviewed:
- Description: This paper is concerned with the general polynomial programming problem with box constraints, including global optimality conditions and optimization methods. First, a necessary global optimality condition for a general polynomial programming problem with box constraints is given. Then we design a local optimization method by using the necessary global optimality condition to obtain some strongly or -strongly local minimizers which substantially improve some KKT points. Finally, a global optimization method, by combining the new local optimization method and an auxiliary function, is designed. Numerical examples show that our methods are efficient and stable.
Solving DC programs using the cutting angle method
- Authors: Ferrer, Albert , Bagirov, Adil , Beliakov, Gleb
- Date: 2015
- Type: Text , Journal article
- Relation: Journal of Global Optimization Vol. 61, no. 1 (2015), p. 71-89
- Relation: http://purl.org/au-research/grants/arc/DP140103213
- Full Text: false
- Reviewed:
- Description: In this paper, we propose a new algorithm for global minimization of functions represented as a difference of two convex functions. The proposed method is a derivative free method and it is designed by adapting the extended cutting angle method. We present preliminary results of numerical experiments using test problems with difference of convex objective functions and box-constraints. We also compare the proposed algorithm with a classical one that uses prismatical subdivisions.