Your selections:

19Bagirov, Adil
10Karmitsa, Napsu
10Mammadov, Musa
4Joki, Kaisa
4Mäkelä, Marko
2Brown, Simon
2Gaudioso, Manlio
2Gondal, Iqbal
2Seifollahi, Sattar
1Al Nuaimat, Alia
1Asadi, Soodabeh
1Bai, Fusheng
1Chi, Chihung
1Cimen, Emre
1Harkness, Greg
1Makela, Marko
1Makinen, Pauliina
1Sultanova, Nargiz
1Ugon, Julien

Show More

Show Less

10Nonsmooth optimization
70102 Applied Mathematics
60103 Numerical and Computational Mathematics
6Nonconvex optimization
40801 Artificial Intelligence and Image Processing
4Bayesian networks
4Bundle methods
4DC optimization
4Optimization
308 Information and Computing Sciences
30906 Electrical and Electronic Engineering
3Cluster analysis
3Data mining
3Incremental approach
3Line search
2Algorithms
2Classification
2Clusterwise linear regression
2DC functions
2Descent direction

Show More

Show Less

Format Type

Limited Memory Bundle Method for Clusterwise Linear Regression

- Karmitsa, Napsu, Bagirov, Adil, Taheri, Sona, Joki, Kaisa

**Authors:**Karmitsa, Napsu , Bagirov, Adil , Taheri, Sona , Joki, Kaisa**Date:**2022**Type:**Text , Book chapter , Book Chapter**Relation:**Intelligent Systems, Control and Automation: Science and Engineering p. 109-122**Full Text:**false**Reviewed:****Description:**A clusterwise linear regression problem consists of finding a number of linear functions each approximating a subset of the given data. In this paper, the limited memory bundle method is modified and combined with the incremental approach to solve this problem using its nonsmooth optimization formulation. The main contribution of the proposed method is to obtain a fast solution time for large-scale clusterwise linear regression problems. The proposed algorithm is tested on small and large real-world data sets and compared with other algorithms for clusterwise linear regression. Numerical results demonstrate that the proposed algorithm is especially efficient in data sets with large numbers of data points and input variables. © 2022, Springer Nature Switzerland AG.

Aggregate subgradient method for nonsmooth DC optimization

- Bagirov, Adil, Taheri, Sona, Joki, Kaisa, Karmitsa, Napsu, Mäkelä, Marko

**Authors:**Bagirov, Adil , Taheri, Sona , Joki, Kaisa , Karmitsa, Napsu , Mäkelä, Marko**Date:**2021**Type:**Text , Journal article**Relation:**Optimization Letters Vol. 15, no. 1 (2021), p. 83-96**Relation:**http://purl.org/au-research/grants/arc/DP190100580**Full Text:**false**Reviewed:****Description:**The aggregate subgradient method is developed for solving unconstrained nonsmooth difference of convex (DC) optimization problems. The proposed method shares some similarities with both the subgradient and the bundle methods. Aggregate subgradients are defined as a convex combination of subgradients computed at null steps between two serious steps. At each iteration search directions are found using only two subgradients: the aggregate subgradient and a subgradient computed at the current null step. It is proved that the proposed method converges to a critical point of the DC optimization problem and also that the number of null steps between two serious steps is finite. The new method is tested using some academic test problems and compared with several other nonsmooth DC optimization solvers. © 2020, Springer-Verlag GmbH Germany, part of Springer Nature.

Incremental DC optimization algorithm for large-scale clusterwise linear regression

- Bagirov, Adil, Taheri, Sona, Cimen, Emre

**Authors:**Bagirov, Adil , Taheri, Sona , Cimen, Emre**Date:**2021**Type:**Text , Journal article**Relation:**Journal of Computational and Applied Mathematics Vol. 389, no. (2021), p. 1-17**Relation:**https://purl.org/au-research/grants/arc/DP190100580**Full Text:**false**Reviewed:****Description:**The objective function in the nonsmooth optimization model of the clusterwise linear regression (CLR) problem with the squared regression error is represented as a difference of two convex functions. Then using the difference of convex algorithm (DCA) approach the CLR problem is replaced by the sequence of smooth unconstrained optimization subproblems. A new algorithm based on the DCA and the incremental approach is designed to solve the CLR problem. We apply the Quasi-Newton method to solve the subproblems. The proposed algorithm is evaluated using several synthetic and real-world data sets for regression and compared with other algorithms for CLR. Results demonstrate that the DCA based algorithm is efficient for solving CLR problems with the large number of data points and in particular, outperforms other algorithms when the number of input variables is small. © 2020 Elsevier B.V.

Clusterwise support vector linear regression

- Joki, Kaisa, Bagirov, Adil, Karmitsa, Napsu, Mäkelä, Marko, Taheri, Sona

**Authors:**Joki, Kaisa , Bagirov, Adil , Karmitsa, Napsu , Mäkelä, Marko , Taheri, Sona**Date:**2020**Type:**Text , Journal article**Relation:**European Journal of Operational Research Vol. 287, no. 1 (2020), p. 19-35**Full Text:**false**Reviewed:****Description:**In clusterwise linear regression (CLR), the aim is to simultaneously partition data into a given number of clusters and to find regression coefficients for each cluster. In this paper, we propose a novel approach to model and solve the CLR problem. The main idea is to utilize the support vector machine (SVM) approach to model the CLR problem by using the SVM for regression to approximate each cluster. This new formulation of the CLR problem is represented as an unconstrained nonsmooth optimization problem, where we minimize a difference of two convex (DC) functions. To solve this problem, a method based on the combination of the incremental algorithm and the double bundle method for DC optimization is designed. Numerical experiments are performed to validate the reliability of the new formulation for CLR and the efficiency of the proposed method. The results show that the SVM approach is suitable for solving CLR problems, especially, when there are outliers in data. © 2020 Elsevier B.V.**Description:**Funding details: Academy of Finland, 289500, 294002, 319274 Funding details: Turun Yliopisto Funding details: Australian Research Council, ARC, (Project no. DP190100580 ).

Cyberattack triage using incremental clustering for intrusion detection systems

- Taheri, Sona, Bagirov, Adil, Gondal, Iqbal, Brown, Simon

**Authors:**Taheri, Sona , Bagirov, Adil , Gondal, Iqbal , Brown, Simon**Date:**2020**Type:**Text , Journal article**Relation:**International Journal of Information Security Vol. 19, no. 5 (2020), p. 597-607**Relation:**http://purl.org/au-research/grants/arc/DP190100580**Full Text:**false**Reviewed:****Description:**Intrusion detection systems (IDSs) are devices or software applications that monitor networks or systems for malicious activities and signals alerts/alarms when such activity is discovered. However, an IDS may generate many false alerts which affect its accuracy. In this paper, we develop a cyberattack triage algorithm to detect these alerts (so-called outliers). The proposed algorithm is designed using the clustering, optimization and distance-based approaches. An optimization-based incremental clustering algorithm is proposed to find clusters of different types of cyberattacks. Using a special procedure, a set of clusters is divided into two subsets: normal and stable clusters. Then, outliers are found among stable clusters using an average distance between centroids of normal clusters. The proposed algorithm is evaluated using the well-known IDS data sets—Knowledge Discovery and Data mining Cup 1999 and UNSW-NB15—and compared with some other existing algorithms. Results show that the proposed algorithm has a high detection accuracy and its false negative rate is very low. © 2019, Springer-Verlag GmbH Germany, part of Springer Nature.**Description:**This research was conducted in Internet Commerce Security Laboratory (ICSL) funded by Westpac Banking Corporation Australia. In addition, the research by Dr. Sona Taheri and A/Prof. Adil Bagirov was supported by the Australian Government through the Australian Research Council’s Discovery Projects funding scheme (DP190100580).

- Bagirov, Adil, Taheri, Sona, Karmitsa, Napsu

**Authors:**Bagirov, Adil , Taheri, Sona , Karmitsa, Napsu**Date:**2020**Type:**Text , Book chapter**Relation:**Numerical Nonsmooth Optimization: State of the Art Algorithms p. 621-654**Full Text:**false**Reviewed:****Description:**In this chapter, the notion of a discrete gradient is introduced and it is shown that the discrete gradients can be used to approximate subdifferentials of a broad class of nonsmooth functions. Two methods based on such approximations, more specifically, the discrete gradient method (DGM) and its limited memory version (LDGB), are described. These methods are semi derivative-free methods for solving nonsmooth and, in general, nonconvex optimization problems. The performance of the methods is demonstrated using some academic test problems. © Springer Nature Switzerland AG 2020.

- Bagirov, Adil, Gaudioso, Manlio, Karmitsa, Napsu, Mäkelä, Marko, Taheri, Sona

**Authors:**Bagirov, Adil , Gaudioso, Manlio , Karmitsa, Napsu , Mäkelä, Marko , Taheri, Sona**Date:**2020**Type:**Text , Book chapter**Relation:**Numerical Nonsmooth Optimization: State of the Art Algorithms p. 693-694**Full Text:**false**Reviewed:**

- Bagirov, Adil, Gaudioso, Manlio, Karmitsa, Napsu, Mäkelä, Marko, Taheri, Sona

**Authors:**Bagirov, Adil , Gaudioso, Manlio , Karmitsa, Napsu , Mäkelä, Marko , Taheri, Sona**Date:**2020**Type:**Text , Book chapter**Relation:**Numerical Nonsmooth Optimization: State of the Art Algorithms p. 1-16**Full Text:**false**Reviewed:****Description:**Nonsmooth optimization (NSO) is among the most challenging tasks in the field of mathematical programming. It addresses optimization problems where objective and/or constraint functions have discontinuous gradients. NSO problems arise in many real life applications. Moreover, some smooth optimization techniques like different decomposition methods, dual formulations and exact penalty methods may lead us to solve NSO problems being either smaller in dimension or simpler in structure. In addition, some optimization problems may be analytically smooth but numerically nonsmooth. This is the case, for instance, with noisy input data and so-called stiff problems, which are numerically unstable and behave like nonsmooth problems. © Springer Nature Switzerland AG 2020.

Missing value imputation via clusterwise linear regression

- Karmitsa, Napsu, Taheri, Sona, Bagirov, Adil, Makinen, Pauliina

**Authors:**Karmitsa, Napsu , Taheri, Sona , Bagirov, Adil , Makinen, Pauliina**Date:**2020**Type:**Text , Journal article**Relation:**IEEE transactions on knowledge and data engineering Vol. , no. (2020), p. 1-1**Full Text:**false**Reviewed:****Description:**In this paper a new method of preprocessing incomplete data is introduced. The method is based on clusterwise linear regression and it combines two well-known approaches for missing value imputation: linear regression and clustering. The idea is to approximate missing values using only those data points that are somewhat similar to the incomplete data point. A similar idea is used also in clustering based imputation methods. Nevertheless, here the linear regression approach is used within each cluster to accurately predict the missing values, and this is done simultaneously to clustering. The proposed method is tested using some synthetic and real-world data sets and compared with other algorithms for missing value imputations. Numerical results demonstrate that the proposed method produces the most accurate imputations in MCAR and MAR data sets with a clear structure and the percentages of missing data no more than 25%

A difference of convex optimization algorithm for piecewise linear regression

- Bagirov, Adil, Taheri, Sona, Asadi, Soodabeh

**Authors:**Bagirov, Adil , Taheri, Sona , Asadi, Soodabeh**Date:**2019**Type:**Text , Journal article**Relation:**Journal of Industrial and Management Optimization Vol. 15, no. 2 (2019), p. 909-932**Relation:**http://purl.org/au-research/grants/arc/DP140103213**Full Text:**false**Reviewed:****Description:**The problem of finding a continuous piecewise linear function approximating a regression function is considered. This problem is formulated as a nonconvex nonsmooth optimization problem where the objective function is represented as a difference of convex (DC) functions. Subdifferentials of DC components are computed and an algorithm is designed based on these subdifferentials to find piecewise linear functions. The algorithm is tested using some synthetic and real world data sets and compared with other regression algorithms.

- Bagirov, Adil, Taheri, Sona, Bai, Fusheng, Wu, Zhiyou

**Authors:**Bagirov, Adil , Taheri, Sona , Bai, Fusheng , Wu, Zhiyou**Date:**2019**Type:**Text , Book chapter**Relation:**Nonsmooth Optimization and Its Applications (part of the International Series of Numerical Mathematics book series) Chapter 2 p. 17-44**Full Text:**false**Reviewed:****Description:**Nonsmooth convex optimization problems with two blocks of variables subject to linear constraints are considered. A new version of the alternating direction method of multipliers is developed for solving these problems. In this method the subproblems are solved approximately. The convergence of the method is studied. New test problems are designed and used to verify the efficiency of the proposed method and to compare it with two versions of the proximal bundle method.

Multi-source cyber-attacks detection using machine learning

- Taheri, Sona, Gondal, Iqbal, Bagirov, Adil, Harkness, Greg, Brown, Simon, Chi, Chihung

**Authors:**Taheri, Sona , Gondal, Iqbal , Bagirov, Adil , Harkness, Greg , Brown, Simon , Chi, Chihung**Date:**2019**Type:**Text , Conference proceedings , Conference paper**Relation:**2019 IEEE International Conference on Industrial Technology, ICIT 2019; Melbourne, Australia; 13th-15th February 2019 Vol. 2019-February, p. 1167-1172**Full Text:****Reviewed:****Description:**The Internet of Things (IoT) has significantly increased the number of devices connected to the Internet ranging from sensors to multi-source data information. As the IoT continues to evolve with new technologies number of threats and attacks against IoT devices are on the increase. Analyzing and detecting these attacks originating from different sources needs machine learning models. These models provide proactive solutions for detecting attacks and their sources. In this paper, we propose to apply a supervised machine learning classification technique to identify cyber-attacks from each source. More precisely, we apply the incremental piecewise linear classifier that constructs boundary between sources/classes incrementally starting with one hyperplane and adding more hyperplanes at each iteration. The algorithm terminates when no further significant improvement of the separation of sources/classes is possible. The construction and usage of piecewise linear boundaries allows us to avoid any possible overfitting. We apply the incremental piecewise linear classifier on the multi-source real world cyber security data set to identify cyber-attacks and their sources.**Description:**Proceedings of the IEEE International Conference on Industrial Technology

**Authors:**Taheri, Sona , Gondal, Iqbal , Bagirov, Adil , Harkness, Greg , Brown, Simon , Chi, Chihung**Date:**2019**Type:**Text , Conference proceedings , Conference paper**Relation:**2019 IEEE International Conference on Industrial Technology, ICIT 2019; Melbourne, Australia; 13th-15th February 2019 Vol. 2019-February, p. 1167-1172**Full Text:****Reviewed:****Description:**The Internet of Things (IoT) has significantly increased the number of devices connected to the Internet ranging from sensors to multi-source data information. As the IoT continues to evolve with new technologies number of threats and attacks against IoT devices are on the increase. Analyzing and detecting these attacks originating from different sources needs machine learning models. These models provide proactive solutions for detecting attacks and their sources. In this paper, we propose to apply a supervised machine learning classification technique to identify cyber-attacks from each source. More precisely, we apply the incremental piecewise linear classifier that constructs boundary between sources/classes incrementally starting with one hyperplane and adding more hyperplanes at each iteration. The algorithm terminates when no further significant improvement of the separation of sources/classes is possible. The construction and usage of piecewise linear boundaries allows us to avoid any possible overfitting. We apply the incremental piecewise linear classifier on the multi-source real world cyber security data set to identify cyber-attacks and their sources.**Description:**Proceedings of the IEEE International Conference on Industrial Technology

Clustering in large data sets with the limited memory bundle method

- Karmitsa, Napsu, Bagirov, Adil, Taheri, Sona

**Authors:**Karmitsa, Napsu , Bagirov, Adil , Taheri, Sona**Date:**2018**Type:**Text , Journal article**Relation:**Pattern Recognition Vol. 83, no. (2018), p. 245-259**Relation:**http://purl.org/au-research/grants/arc/DP140103213**Full Text:**false**Reviewed:****Description:**The aim of this paper is to design an algorithm based on nonsmooth optimization techniques to solve the minimum sum-of-squares clustering problems in very large data sets. First, the clustering problem is formulated as a nonsmooth optimization problem. Then the limited memory bundle method [Haarala et al., 2007] is modified and combined with an incremental approach to design a new clustering algorithm. The algorithm is evaluated using real world data sets with both the large number of attributes and the large number of data points. It is also compared with some other optimization based clustering algorithms. The numerical results demonstrate the efficiency of the proposed algorithm for clustering in very large data sets.

Double bundle method for finding clarke stationary points in nonsmooth dc programming

- Joki, Kaisa, Bagirov, Adil, Karmitsa, Napsu, Makela, Marko, Taheri, Sona

**Authors:**Joki, Kaisa , Bagirov, Adil , Karmitsa, Napsu , Makela, Marko , Taheri, Sona**Date:**2018**Type:**Text , Journal article**Relation:**SIAM Journal on Optimization Vol. 28, no. 2 (2018), p. 1892-1919**Relation:**http://purl.org/au-research/grants/arc/DP140103213**Full Text:****Reviewed:****Description:**The aim of this paper is to introduce a new proximal double bundle method for unconstrained nonsmooth optimization, where the objective function is presented as a difference of two convex (DC) functions. The novelty in our method is a new escape procedure which enables us to guarantee approximate Clarke stationarity for solutions by utilizing the DC components of the objective function. This optimality condition is stronger than the criticality condition typically used in DC programming. Moreover, if a candidate solution is not approximate Clarke stationary, then the escape procedure returns a descent direction. With this escape procedure, we can avoid some shortcomings encountered when criticality is used. The finite termination of the double bundle method to an approximate Clarke stationary point is proved by assuming that the subdifferentials of DC components are polytopes. Finally, some encouraging numerical results are presented.

**Authors:**Joki, Kaisa , Bagirov, Adil , Karmitsa, Napsu , Makela, Marko , Taheri, Sona**Date:**2018**Type:**Text , Journal article**Relation:**SIAM Journal on Optimization Vol. 28, no. 2 (2018), p. 1892-1919**Relation:**http://purl.org/au-research/grants/arc/DP140103213**Full Text:****Reviewed:****Description:**The aim of this paper is to introduce a new proximal double bundle method for unconstrained nonsmooth optimization, where the objective function is presented as a difference of two convex (DC) functions. The novelty in our method is a new escape procedure which enables us to guarantee approximate Clarke stationarity for solutions by utilizing the DC components of the objective function. This optimality condition is stronger than the criticality condition typically used in DC programming. Moreover, if a candidate solution is not approximate Clarke stationary, then the escape procedure returns a descent direction. With this escape procedure, we can avoid some shortcomings encountered when criticality is used. The finite termination of the double bundle method to an approximate Clarke stationary point is proved by assuming that the subdifferentials of DC components are polytopes. Finally, some encouraging numerical results are presented.

Solving minimax problems : Local smoothing versus global smoothing

- Bagirov, Adil, Sultanova, Nargiz, Al Nuaimat, Alia, Taheri, Sona

**Authors:**Bagirov, Adil , Sultanova, Nargiz , Al Nuaimat, Alia , Taheri, Sona**Date:**2018**Type:**Text , Conference proceedings**Relation:**4th International Conference on Numerical Analysis and Optimization, NAO-IV 2017; Muscat, Oman; 2nd-5th January 2017; published in Numerical Analysis and Optimization NAO-IV (part of the Springer Proceedings in Mathematics and Statistics book series PROMS, volume 235) Vol. 235, p. 23-43**Full Text:**false**Reviewed:****Description:**The aim of this chapter is to compare different smoothing techniques for solving finite minimax problems. We consider the local smoothing technique which approximates the function in some neighborhood of a point of nondifferentiability and also global smoothing techniques such as the exponential and hyperbolic smoothing which approximate the function in the whole domain. Computational results on the collection of academic test problems are used to compare different smoothing techniques. Results show the superiority of the local smoothing technique for convex problems and global smoothing techniques for nonconvex problems. © 2018, Springer International Publishing AG, part of Springer Nature.**Description:**Springer Proceedings in Mathematics and Statistics

DC programming algorithm for clusterwise linear L1 regression

**Authors:**Bagirov, Adil , Taheri, Sona**Date:**2017**Type:**Text , Journal article**Relation:**Journal of the Operations Research Society of China Vol. 5, no. 2 (2017), p. 233-256**Relation:**http://purl.org/au-research/grants/arc/DP140103213**Full Text:**false**Reviewed:****Description:**The aim of this paper is to develop an algorithm for solving the clusterwise linear least absolute deviations regression problem. This problem is formulated as a nonsmooth nonconvex optimization problem, and the objective function is represented as a difference of convex functions. Optimality conditions are derived by using this representation. An algorithm is designed based on the difference of convex representation and an incremental approach. The proposed algorithm is tested using small to large artificial and real-world data sets. © 2017, Operations Research Society of China, Periodicals Agency of Shanghai University, Science Press, and Springer-Verlag Berlin Heidelberg.

New diagonal bundle method for clustering problems in large data sets

- Karmitsa, Napsu, Bagirov, Adil, Taheri, Sona

**Authors:**Karmitsa, Napsu , Bagirov, Adil , Taheri, Sona**Date:**2017**Type:**Text , Journal article**Relation:**European Journal of Operational Research Vol. 263, no. 2 (2017), p. 367-379**Relation:**http://purl.org/au-research/grants/arc/DP140103213**Full Text:**false**Reviewed:****Description:**Clustering is one of the most important tasks in data mining. Recent developments in computer hardware allow us to store in random access memory (RAM) and repeatedly read data sets with hundreds of thousands and even millions of data points. This makes it possible to use conventional clustering algorithms in such data sets. However, these algorithms may need prohibitively large computational time and fail to produce accurate solutions. Therefore, it is important to develop clustering algorithms which are accurate and can provide real time clustering in large data sets. This paper introduces one of them. Using nonsmooth optimization formulation of the clustering problem the objective function is represented as a difference of two convex (DC) functions. Then a new diagonal bundle algorithm that explicitly uses this structure is designed and combined with an incremental approach to solve this problem. The method is evaluated using real world data sets with both large number of attributes and large number of data points. The proposed method is compared with two other clustering algorithms using numerical results. © 2017 Elsevier B.V.

Nonsmooth DC programming approach to the minimum sum-of-squares clustering problems

- Bagirov, Adil, Taheri, Sona, Ugon, Julien

**Authors:**Bagirov, Adil , Taheri, Sona , Ugon, Julien**Date:**2016**Type:**Text , Journal article**Relation:**Pattern Recognition Vol. 53, no. (2016), p. 12-24**Relation:**http://purl.org/au-research/grants/arc/DP140103213**Full Text:**false**Reviewed:****Description:**This paper introduces an algorithm for solving the minimum sum-of-squares clustering problems using their difference of convex representations. A non-smooth non-convex optimization formulation of the clustering problem is used to design the algorithm. Characterizations of critical points, stationary points in the sense of generalized gradients and inf-stationary points of the clustering problem are given. The proposed algorithm is tested and compared with other clustering algorithms using large real world data sets. © 2015 Elsevier Ltd. All rights reserved.

Structure learning of Bayesian Networks using global optimization with applications in data classification

- Taheri, Sona, Mammadov, Musa

**Authors:**Taheri, Sona , Mammadov, Musa**Date:**2014**Type:**Text , Journal article**Relation:**Optimization Letters Vol. 9, no. 5 (2014), p. 931-948**Full Text:****Reviewed:****Description:**Bayesian Networks are increasingly popular methods of modeling uncertainty in artificial intelligence and machine learning. A Bayesian Network consists of a directed acyclic graph in which each node represents a variable and each arc represents probabilistic dependency between two variables. Constructing a Bayesian Network from data is a learning process that consists of two steps: learning structure and learning parameter. Learning a network structure from data is the most difficult task in this process. This paper presents a new algorithm for constructing an optimal structure for Bayesian Networks based on optimization. The algorithm has two major parts. First, we define an optimization model to find the better network graphs. Then, we apply an optimization approach for removing possible cycles from the directed graphs obtained in the first part which is the first of its kind in the literature. The main advantage of the proposed method is that the maximal number of parents for variables is not fixed a priory and it is defined during the optimization procedure. It also considers all networks including cyclic ones and then choose a best structure by applying a global optimization method. To show the efficiency of the algorithm, several closely related algorithms including unrestricted dependency Bayesian Network algorithm, as well as, benchmarks algorithms SVM and C4.5 are employed for comparison. We apply these algorithms on data classification; data sets are taken from the UCI machine learning repository and the LIBSVM. © 2014, Springer-Verlag Berlin Heidelberg.

**Authors:**Taheri, Sona , Mammadov, Musa**Date:**2014**Type:**Text , Journal article**Relation:**Optimization Letters Vol. 9, no. 5 (2014), p. 931-948**Full Text:****Reviewed:****Description:**Bayesian Networks are increasingly popular methods of modeling uncertainty in artificial intelligence and machine learning. A Bayesian Network consists of a directed acyclic graph in which each node represents a variable and each arc represents probabilistic dependency between two variables. Constructing a Bayesian Network from data is a learning process that consists of two steps: learning structure and learning parameter. Learning a network structure from data is the most difficult task in this process. This paper presents a new algorithm for constructing an optimal structure for Bayesian Networks based on optimization. The algorithm has two major parts. First, we define an optimization model to find the better network graphs. Then, we apply an optimization approach for removing possible cycles from the directed graphs obtained in the first part which is the first of its kind in the literature. The main advantage of the proposed method is that the maximal number of parents for variables is not fixed a priory and it is defined during the optimization procedure. It also considers all networks including cyclic ones and then choose a best structure by applying a global optimization method. To show the efficiency of the algorithm, several closely related algorithms including unrestricted dependency Bayesian Network algorithm, as well as, benchmarks algorithms SVM and C4.5 are employed for comparison. We apply these algorithms on data classification; data sets are taken from the UCI machine learning repository and the LIBSVM. © 2014, Springer-Verlag Berlin Heidelberg.

Attribute weighted Naive Bayes classifier using a local optimization

- Taheri, Sona, Yearwood, John, Mammadov, Musa, Seifollahi, Sattar

**Authors:**Taheri, Sona , Yearwood, John , Mammadov, Musa , Seifollahi, Sattar**Date:**2013**Type:**Text , Journal article**Relation:**Neural Computing & Applications Vol.24, no.5 (2013), p.995-1002**Full Text:****Reviewed:****Description:**The Naive Bayes classifier is a popular classification technique for data mining and machine learning. It has been shown to be very effective on a variety of data classification problems. However, the strong assumption that all attributes are conditionally independent given the class is often violated in real-world applications. Numerous methods have been proposed in order to improve the performance of the Naive Bayes classifier by alleviating the attribute independence assumption. However, violation of the independence assumption can increase the expected error. Another alternative is assigning the weights for attributes. In this paper, we propose a novel attribute weighted Naive Bayes classifier by considering weights to the conditional probabilities. An objective function is modeled and taken into account, which is based on the structure of the Naive Bayes classifier and the attribute weights. The optimal weights are determined by a local optimization method using the quasisecant method. In the proposed approach, the Naive Bayes classifier is taken as a starting point. We report the results of numerical experiments on several real-world data sets in binary classification, which show the efficiency of the proposed method.

**Authors:**Taheri, Sona , Yearwood, John , Mammadov, Musa , Seifollahi, Sattar**Date:**2013**Type:**Text , Journal article**Relation:**Neural Computing & Applications Vol.24, no.5 (2013), p.995-1002**Full Text:****Reviewed:****Description:**The Naive Bayes classifier is a popular classification technique for data mining and machine learning. It has been shown to be very effective on a variety of data classification problems. However, the strong assumption that all attributes are conditionally independent given the class is often violated in real-world applications. Numerous methods have been proposed in order to improve the performance of the Naive Bayes classifier by alleviating the attribute independence assumption. However, violation of the independence assumption can increase the expected error. Another alternative is assigning the weights for attributes. In this paper, we propose a novel attribute weighted Naive Bayes classifier by considering weights to the conditional probabilities. An objective function is modeled and taken into account, which is based on the structure of the Naive Bayes classifier and the attribute weights. The optimal weights are determined by a local optimization method using the quasisecant method. In the proposed approach, the Naive Bayes classifier is taken as a starting point. We report the results of numerical experiments on several real-world data sets in binary classification, which show the efficiency of the proposed method.

Are you sure you would like to clear your session, including search history and login status?