Description:
Determining the architecture and parameters of neural networks is an important scientific challenge. This paper reports a new hybrid optimization method for optimization of back-propagation neural networks architecture and parameters with a high accuracy. We use particle swarm optimization that has proven to be very effective and fast and has shown to increase the efficiency of simulated annealing when applied to a diverse set of optimization problems. To evaluate the proposed method, we employ the PIMA dataset from the University of California machine learning database. Compared with previous work, we show superior classification accuracy rates of the developed approach.
Description:
Determining the architecture and parameters of neural networks is an important scientific challenge. This paper reports a new hybrid optimization method for optimization of back-propagation neural networks architecture and parameters with a high accuracy. We use particle swarm optimization that has proven to be very effective and fast and has shown to increase the efficiency of simulated annealing when applied to a diverse set of optimization problems. To evaluate the proposed method, we employ the PIMA dataset from the University of California machine learning database. Compared with previous work, we show superior classification accuracy rates of the developed approach.
Description:
In this paper we propose the use of optimization based clustering algorithms to determine hierarchical multicast trees. This problem is formulated as an optimization problem with a non-smooth, non-convex objective function. Different algorithms are examined for solving this problem. Results of numerical experiments using some artificial and real-world databases are reported. We compare several optimization based clustering methods and their combinations with the k- means method. The results demonstrate the effectiveness of these algorithms.
Description:
In this paper we propose the use of optimization based clustering algorithms to determine hierarchical multicast trees. This problem is formulated as an optimization problem with a non-smooth, non-convex objective function. Different algorithms are examined for solving this problem. Results of numerical experiments using some artificial and real-world databases are reported. We compare several optimization based clustering methods and their combinations with the k- means method. The results demonstrate the effectiveness of these algorithms.
Description:
Slug flow induces vibration in pipelines, which may, in some cases, result in fatigue failure. This can result from dynamic stresses, induced by the deflection and bending moment in the pipe span, growing to levels above the endurance limits of the pipeline material. As such, it is of paramount importance to understand and quantify the size of the pipeline response to slug flow under given speed and damping conditions. This paper utilizes the results of an optimization procedure to devise a surrogate closed-form model, which can be employed to calculate the maximum values of the pipeline loadings at given values of speed and damping parameters. The surrogate model is intended to replace the computationally costly numerical procedure needed for the analysis. The maximum values of the lateral deflection and bending moment, along with their locations, have been calculated using the optimization method of stochastic perturbation and successive approximations ( SPSA). The accuracy of the proposed surrogate model will be validated numerically, and the model will be subsequently used in a numerical example to demonstrate its applicability in industrial situations. An accompanying spreadsheet with this worked example is also given.
Description:
This paper presents a system dynamic modeling technique for reliability centered maintenance (RCM). It provides a simulation model to analyze the impact of maintenance strategies such as preventive and corrective maintenance on system availability. Also, maintenance cost is associated with the model to have an optimization view for a manufacturing system. As a result, the optimal decisions can be made based on the results of the model. In addition, sensitivity analysis has been conducted for various parameters in the simulation model. The results were compared and discussed for various conditions.
Description:
Naive Bayes is among the simplest probabilistic classifiers. It often performs surprisingly well in many real world applications, despite the strong assumption that all features are conditionally independent given the class. In the learning process of this classifier with the known structure, class probabilities and conditional probabilities are calculated using training data, and then values of these probabilities are used to classify new observations. In this paper, we introduce three novel optimization models for the naive Bayes classifier where both class probabilities and conditional probabilities are considered as variables. The values of these variables are found by solving the corresponding optimization problems. Numerical experiments are conducted on several real world binary classification data sets, where continuous features are discretized by applying three different methods. The performances of these models are compared with the naive Bayes classifier, tree augmented naive Bayes, the SVM, C4.5 and the nearest neighbor classifier. The obtained results demonstrate that the proposed models can significantly improve the performance of the naive Bayes classifier, yet at the same time maintain its simple structure.
Description:
Naive Bayes is among the simplest probabilistic classifiers. It often performs surprisingly well in many real world applications, despite the strong assumption that all features are conditionally independent given the class. In the learning process of this classifier with the known structure, class probabilities and conditional probabilities are calculated using training data, and then values of these probabilities are used to classify new observations. In this paper, we introduce three novel optimization models for the naive Bayes classifier where both class probabilities and conditional probabilities are considered as variables. The values of these variables are found by solving the corresponding optimization problems. Numerical experiments are conducted on several real world binary classification data sets, where continuous features are discretized by applying three different methods. The performances of these models are compared with the naive Bayes classifier, tree augmented naive Bayes, the SVM, C4.5 and the nearest neighbor classifier. The obtained results demonstrate that the proposed models can significantly improve the performance of the naive Bayes classifier, yet at the same time maintain its simple structure.
Description:
An Electrocardiogram (ECG) records the electrical impulses of the heart and indicates rhythm anomalies for diagnostic purposes [1], [2]. A typical ECG tracing of the cardiac cycle consists of a P wave, QRS complex, and T wave [3]. Good performance of an ECG analyzing system depends heavily upon the accurate and reliable detection of the QRS complex, as well as the T and P waves [4]. A Bundle Branch Block (BBB) is a delay or obstruction along electrical impulse pathways of the heart manifesting in a prolonged QRS interval usually greater than 120ms. The automated detection and classification of a BBB is important for prompt, accurate diagnosis and treatment to reduce morbidity and mortality.
Description:
Many optimization problems related to integrated oil and gas production systems are nonconvex and multimodal. Additionally, apart from the innate nonsmoothness of many optimization problems, nonsmooth functions such as minimum and maximum functions may be used to model flow/pressure controllers and cascade mass in the gas gathering and blending networks. In this paper we study the application of different versions of the derivative free Discrete Gradient Method (DGM) as well as the Lipschitz Global Optimizer (LGO) suite to production optimization in integrated oil and gas production systems and their comparison with various local and global solvers used with the General Algebraic Modeling System (GAMS). Four nonconvex and nonsmooth test cases were constructed from a small but realistic integrated gas production system optimization problem. The derivation of the system of equations for the various test cases is also presented. Results demonstrate that DGM is especially effective for solving nonsmooth optimization problems and its two versions are capable global optimization algorithms. We also demonstrate that LGO solves successfully the presented test (as well as other related real-world) problems.
Description:
Many optimization problems related to integrated oil and gas production systems are nonconvex and multimodal. Additionally, apart from the innate nonsmoothness of many optimization problems, nonsmooth functions such as minimum and maximum functions may be used to model flow/pressure controllers and cascade mass in the gas gathering and blending networks. In this paper we study the application of different versions of the derivative free Discrete Gradient Method (DGM) as well as the Lipschitz Global Optimizer (LGO) suite to production optimization in integrated oil and gas production systems and their comparison with various local and global solvers used with the General Algebraic Modeling System (GAMS). Four nonconvex and nonsmooth test cases were constructed from a small but realistic integrated gas production system optimization problem. The derivation of the system of equations for the various test cases is also presented. Results demonstrate that DGM is especially effective for solving nonsmooth optimization problems and its two versions are capable global optimization algorithms. We also demonstrate that LGO solves successfully the presented test (as well as other related real-world) problems.
Description:
Regularity properties lie at the core of variational analysis because of their importance for stability analysis of optimization and variational problems, constraint qualications, qualication conditions in coderivative and subdierential calculus and convergence analysis of numerical algorithms. The thesis is devoted to investigation of several research questions related to regularity properties in variational analysis and their applications in convergence analysis and optimization. Following the works by Kruger, we examine several useful regularity properties of collections of sets in both linear and Holder-type settings and establish their characterizations and relationships to regularity properties of set-valued mappings. Following the recent publications by Lewis, Luke, Malick (2009), Drusvyatskiy, Ioe, Lewis (2014) and some others, we study application of the uniform regularity and related properties of collections of sets to alternating projections for solving nonconvex feasibility problems and compare existing results on this topic. Motivated by Ioe (2000) and his subsequent publications, we use the classical iteration scheme going back to Banach, Schauder, Lyusternik and Graves to establish criteria for regularity properties of set-valued mappings and compare this approach with the one based on the Ekeland variational principle. Finally, following the recent works by Khanh et al. on stability analysis for optimization related problems, we investigate calmness of set-valued solution mappings of variational problems.
Description:
Doctor of Philosophy
Description:
Regularity properties lie at the core of variational analysis because of their importance for stability analysis of optimization and variational problems, constraint qualications, qualication conditions in coderivative and subdierential calculus and convergence analysis of numerical algorithms. The thesis is devoted to investigation of several research questions related to regularity properties in variational analysis and their applications in convergence analysis and optimization. Following the works by Kruger, we examine several useful regularity properties of collections of sets in both linear and H
Description:
Regularity properties lie at the core of variational analysis because of their importance for stability analysis of optimization and variational problems, constraint qualications, qualication conditions in coderivative and subdierential calculus and convergence analysis of numerical algorithms. The thesis is devoted to investigation of several research questions related to regularity properties in variational analysis and their applications in convergence analysis and optimization. Following the works by Kruger, we examine several useful regularity properties of collections of sets in both linear and Holder-type settings and establish their characterizations and relationships to regularity properties of set-valued mappings. Following the recent publications by Lewis, Luke, Malick (2009), Drusvyatskiy, Ioe, Lewis (2014) and some others, we study application of the uniform regularity and related properties of collections of sets to alternating projections for solving nonconvex feasibility problems and compare existing results on this topic. Motivated by Ioe (2000) and his subsequent publications, we use the classical iteration scheme going back to Banach, Schauder, Lyusternik and Graves to establish criteria for regularity properties of set-valued mappings and compare this approach with the one based on the Ekeland variational principle. Finally, following the recent works by Khanh et al. on stability analysis for optimization related problems, we investigate calmness of set-valued solution mappings of variational problems.
Description:
Doctor of Philosophy
Description:
Regularity properties lie at the core of variational analysis because of their importance for stability analysis of optimization and variational problems, constraint qualications, qualication conditions in coderivative and subdierential calculus and convergence analysis of numerical algorithms. The thesis is devoted to investigation of several research questions related to regularity properties in variational analysis and their applications in convergence analysis and optimization. Following the works by Kruger, we examine several useful regularity properties of collections of sets in both linear and H
Description:
The polynomial programming problem which has a polynomial objective function, either with no constraints or with polynomial constraints occurs frequently in engineering design, investment science, control theory, network distribution, signal processing and locationallocation contexts. Moreover, the polynomial programming problem is known to be Nondeterministic Polynomial-time hard (NP-hard). The polynomial programming problem has attracted a lot of attention, including quadratic, cubic, homogenous or normal quartic programming problems as special cases. Existing methods for solving polynomial programming problems include algebraic methods and various convex relaxation methods. Especially, among these methods, semidefinite programming (SDP) and sum of squares (SOS) relaxations are very popular. Theoretically, SDP and SOS relaxation methods are very powerful and successful in solving the general polynomial programming problem with a compact feasible region. However, the solvability in practice depends on the size or the degree of the polynomial programming problem and the required accuracy. Hence, solving large scale SDP problems still remains a computational challenge. It is well-known that traditional local optimization methods are designed based on necessary local optimality conditions, i.e., Karush-Kuhn-Tucker (KKT) conditions. Motivated by this, some researchers proposed a necessary global optimality condition for a quadratic programming problem and designed a new local optimization method according to the necessary global optimality condition. In this thesis, we try to apply this idea to cubic and quatic programming problems, and further to general unconstrained and constrained polynomial programming problems. For these polynomial programming problems, we will investigate necessary global optimality conditions and design new local optimization methods according to these conditions. These necessary global optimality conditions are generally stronger than KKT conditions. Hence, the obtained new local minimizers by using the new local optimization methods may improve some KKT points. Our ultimate aim is to design global optimization methods for these polynomial programming problems. We notice that the filled function method is one of the well-known and practical auxiliary function methods used to achieve a global minimizer. In this thesis, we design global optimization methods by combining the new proposed local optimization methods and some auxiliary functions. The numerical examples illustrate the efficiency and stability of the optimization methods. Finally, we discuss some applications for solving some sensor network localization problems and systems of polynomial equations. It is worth mentioning that we apply the idea and the results for polynomial programming problems to nonlinear programming problems (NLP). We provide an optimality condition and design new local optimization methods according to the optimality condition and design global optimization methods for the problem (NLP) by combining the new local optimization methods and an auxiliary function. In order to test the performance of the global optimization methods, we compare them with two other heuristic methods. The results demonstrate our methods outperform the two other algorithms.
Description:
The polynomial programming problem which has a polynomial objective function, either with no constraints or with polynomial constraints occurs frequently in engineering design, investment science, control theory, network distribution, signal processing and locationallocation contexts. Moreover, the polynomial programming problem is known to be Nondeterministic Polynomial-time hard (NP-hard). The polynomial programming problem has attracted a lot of attention, including quadratic, cubic, homogenous or normal quartic programming problems as special cases. Existing methods for solving polynomial programming problems include algebraic methods and various convex relaxation methods. Especially, among these methods, semidefinite programming (SDP) and sum of squares (SOS) relaxations are very popular. Theoretically, SDP and SOS relaxation methods are very powerful and successful in solving the general polynomial programming problem with a compact feasible region. However, the solvability in practice depends on the size or the degree of the polynomial programming problem and the required accuracy. Hence, solving large scale SDP problems still remains a computational challenge. It is well-known that traditional local optimization methods are designed based on necessary local optimality conditions, i.e., Karush-Kuhn-Tucker (KKT) conditions. Motivated by this, some researchers proposed a necessary global optimality condition for a quadratic programming problem and designed a new local optimization method according to the necessary global optimality condition. In this thesis, we try to apply this idea to cubic and quatic programming problems, and further to general unconstrained and constrained polynomial programming problems. For these polynomial programming problems, we will investigate necessary global optimality conditions and design new local optimization methods according to these conditions. These necessary global optimality conditions are generally stronger than KKT conditions. Hence, the obtained new local minimizers by using the new local optimization methods may improve some KKT points. Our ultimate aim is to design global optimization methods for these polynomial programming problems. We notice that the filled function method is one of the well-known and practical auxiliary function methods used to achieve a global minimizer. In this thesis, we design global optimization methods by combining the new proposed local optimization methods and some auxiliary functions. The numerical examples illustrate the efficiency and stability of the optimization methods. Finally, we discuss some applications for solving some sensor network localization problems and systems of polynomial equations. It is worth mentioning that we apply the idea and the results for polynomial programming problems to nonlinear programming problems (NLP). We provide an optimality condition and design new local optimization methods according to the optimality condition and design global optimization methods for the problem (NLP) by combining the new local optimization methods and an auxiliary function. In order to test the performance of the global optimization methods, we compare them with two other heuristic methods. The results demonstrate our methods outperform the two other algorithms.