Canonical dual solutions to nonconvex radial basis neural network optimization problem
- Authors: Latorre, Vittorio , Gao, David
- Date: 2014
- Type: Text , Journal article
- Relation: Neurocomputing Vol. 134, no. Special issue (2014), p. 189-197
- Full Text: false
- Reviewed:
- Description: Radial Basis Functions Neural Networks (RBFNNs) are tools widely used in regression problems. One of their principal drawbacks is that the formulation corresponding to the training with the supervision of both the centers and the weights is a highly non-convex optimization problem, which leads to some fundamental difficulties for the traditional optimization theory and methods. This paper presents a generalized canonical duality theory for solving this challenging problem. We demonstrate that by using sequential canonical dual transformations, the nonconvex optimization problem of the RBFNN can be reformulated as a canonical dual problem (without duality gap). Both global optimal solution and local extrema can be classified. Several applications to one of the most used Radial Basis Functions, the Gaussian function, are illustrated. Our results show that even for a one-dimensional case, the global minimizer of the nonconvex problem may not be the best solution to the RBFNNs, and the canonical dual theory is a promising tool for solving general neural networks training problems. © 2014 Elsevier B.V.
Canonical duality for radial basis neural networks
- Authors: Latorre, Vittorio , Gao, David
- Date: 2013
- Type: Text , Journal article
- Relation: Advances in Intelligent Systems and Computing Vol. 212, no. (2013), p. 1189-1197
- Full Text: false
- Reviewed:
- Description: Radial Basis Function Neural Networks (RBF NN) are a tool largely used for regression problems. The principal drawback of this kind of predictive tool is that the optimization problem solved to train the network can be non-convex. On the other hand Canonical Duality Theory offers a powerful procedure to reformulate general non-convex problems in dual forms so that it is possible to find optimal solutions and to get deep insights into the nature of the challenging problems. By combining the canonical duality theory with the RBF NN, this paper presents a potentially useful method for solving challenging problems in real-world applications. © Springer-Verlag Berlin Heidelberg 2013. Proceedings of the Eighth International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA), 2013.
- Description: 2003011221
Canonical duality for solving general nonconvex constrained problems
- Authors: Latorre, Vittorio , Gao, David
- Date: 2016
- Type: Text , Journal article
- Relation: Optimization Letters Vol. 10, no. 8 (2016), p. 1763-1779
- Full Text:
- Reviewed:
- Description: This paper presents a canonical duality theory for solving a general nonconvex constrained optimization problem within a unified framework to cover Lagrange multiplier method and KKT theory. It is proved that if both target function and constraints possess certain patterns necessary for modeling real systems, a perfect dual problem (without duality gap) can be obtained in a unified form with global optimality conditions provided.While the popular augmented Lagrangian method may produce more difficult nonconvex problems due to the nonlinearity of constraints. Some fundamental concepts such as the objectivity and Lagrangian in nonlinear programming are addressed.
Canonical duality-triality theory: Bridge between nonconvex analysis/mechanics and global optimization in complex system
- Authors: Gao, David , Ruan, Ning , Latorre, Vittorio
- Date: 2017
- Type: Text , Book chapter
- Relation: Canonical duality theory: Unified methodology for multidisciplinary study Chapter 1 p. 1-47
- Full Text: false
- Reviewed:
- Description: Canonical duality-triality is a breakthrough methodological theory, which can be used not only for modeling complex systems within a unified framework, but also for solving a wide class of challenging problems from real-world applications. This paper presents a brief review on this theory, its philosophical origin, physics foundation, and mathematical statements in both finite- and infinite-dimensional spaces. Particular emphasis is placed on its role for bridging the gap between nonconvex analysis/mechanics and global optimization . Special attentions are paid on unified understanding the fundamental difficulties in large deformation mechanics, bifurcation/chaos in nonlinear science, and the NP-hard problems in global optimization, as well as the theorems, methods, and algorithms for solving these challenging problems. Misunderstandings and confusion on some basic concepts, such as objectivity , nonlinearity, Lagrangian , and generalized convexities are discussed and classified. Breakthrough from recent challenges and conceptual mistakes by M. Voisei, C. Z
Efficient deterministic algorithm for huge-sized noisy sensor localization problems via canonical duality theory
- Authors: Latorre, Vittorio , Gao, David
- Date: 2021
- Type: Text , Journal article
- Relation: IEEE Transactions on Cybernetics Vol. 51, no. 10 (2021), p. 5069-5081
- Full Text: false
- Reviewed:
- Description: This paper presents a new deterministic method and a polynomial-time algorithm for solving general huge-sized sensor network localization problems. The problem is first formulated as a nonconvex minimization, which was considered as an NP-hard based on conventional theories. However, by the canonical duality theory, this challenging problem can be equivalently converted into a convex dual problem. By introducing a new optimality measure, a powerful canonical primal-dual interior (CPDI) point algorithm is developed which can solve efficiently huge-sized problems with hundreds of thousands of sensors. The new method is compared with the popular methods in the literature. Results show that the CPDI algorithm is not only faster than the benchmarks but also much more accurate on networks affected by noise on the distances. © 2013 IEEE.
Global optimal trajectory in Chaos and NP-Hardness
- Authors: Latorre, Vittorio , Gao, David
- Date: 2016
- Type: Text , Journal article
- Relation: International Journal of Bifurcation and Chaos Vol. 26, no. 8 (2016), p. 1-14
- Full Text:
- Reviewed:
- Description: This paper presents an unconventional theory and method for solving general nonlinear dynamical systems. Instead of the direct iterative methods, the discretized nonlinear system is first formulated as a global optimization problem via the least squares method. A newly developed canonical duality theory shows that this nonconvex minimization problem can be solved deterministically in polynomial time if a global optimality condition is satisfied. The so-called pseudo-chaos produced by linear iterative methods are mainly due to the intrinsic numerical error accumulations. Otherwise, the global optimization problem could be NP-hard and the nonlinear system can be really chaotic. A conjecture is proposed, which reveals the connection between chaos in nonlinear dynamics and NP-hardness in computer science. The methodology and the conjecture are verified by applications to the well-known logistic equation, a forced memristive circuit and the Lorenz system. Computational results show that the canonical duality theory can be used to identify chaotic systems and to obtain realistic global optimal solutions in nonlinear dynamical systems. The method and results presented in this paper should bring some new insights into nonlinear dynamical systems and NP-hardness in computational complexity theory. © 2016 World Scientific Publishing Company.