Hybrid intrusion detection system based on the stacking ensemble of C5 decision tree classifier and one class support vector machine
- Authors: Khraisat, Ansam , Gondal, Iqbal , Vamplew, Peter , Kamruzzaman, Joarder , Alazab, Ammar
- Date: 2020
- Type: Text , Journal article
- Relation: Electronics (Switzerland) Vol. 9, no. 1 (2020), p.
- Full Text:
- Reviewed:
- Description: Cyberttacks are becoming increasingly sophisticated, necessitating the efficient intrusion detection mechanisms to monitor computer resources and generate reports on anomalous or suspicious activities. Many Intrusion Detection Systems (IDSs) use a single classifier for identifying intrusions. Single classifier IDSs are unable to achieve high accuracy and low false alarm rates due to polymorphic, metamorphic, and zero-day behaviors of malware. In this paper, a Hybrid IDS (HIDS) is proposed by combining the C5 decision tree classifier and One Class Support Vector Machine (OC-SVM). HIDS combines the strengths of SIDS) and Anomaly-based Intrusion Detection System (AIDS). The SIDS was developed based on the C5.0 Decision tree classifier and AIDS was developed based on the one-class Support Vector Machine (SVM). This framework aims to identify both the well-known intrusions and zero-day attacks with high detection accuracy and low false-alarm rates. The proposed HIDS is evaluated using the benchmark datasets, namely, Network Security Laboratory-Knowledge Discovery in Databases (NSL-KDD) and Australian Defence Force Academy (ADFA) datasets. Studies show that the performance of HIDS is enhanced, compared to SIDS and AIDS in terms of detection rate and low false-alarm rates. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.
Categorical features transformation with compact one-hot encoder for fraud detection in distributed environment
- Authors: Ul Haq, Ikram , Gondal, Iqbal , Vamplew, Peter , Brown, Simon
- Date: 2019
- Type: Text , Conference proceedings , Conference paper
- Relation: 2019 16th Australasian Conference on Data Mining, AusDM 2018; Bathurst, NSW; 28 November 2018 through 30 November 2018 Vol. 996, p. 69-80
- Full Text: false
- Reviewed:
- Description: Fraud detection for online banking is an important research area, but one of the challenges is the heterogeneous nature of transactions data i.e. a combination of numeric as well as mixed attributes. Usually, numeric format data gives better performance for classification, regression and clustering algorithms. However, many machine learning problems have categorical, or nominal features, rather than numeric features only. In addition, some machine learning platforms such as Apache Spark accept numeric data only. One-hot Encoding (OHE) is a widely used approach for transforming categorical features to numerical features in traditional data mining tasks. The one-hot approach has some challenges as well: the sparseness of the transformed data and that the distinct values of an attribute are not always known in advance. Other than the model accuracy, compactness of machine learning models is equally important due to growing memory and storage needs. This paper presents an innovative technique to transform categorical features to numeric features by compacting sparse data even if all the distinct values are not known. The transformed data can be used for the development of fraud detection systems. The accuracy of the results has been validated on synthetic and real bank fraud data and a publicly available anomaly detection (KDD-99) dataset on a multi-node data cluster. © Springer Nature Singapore Pte Ltd. 2019.
Dependable large scale behavioral patterns mining from sensor data using Hadoop platform
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2017
- Type: Text , Journal article
- Relation: Information Sciences Vol. 379, no. (2017), p. 128-145
- Full Text: false
- Reviewed:
- Description: Wireless sensor networks (WSNs) will be an integral part of the future Internet of Things (loT) environment and generate large volumes of data. However, these data would only be of benefit if useful knowledge can be mined from them. A data mining framework for WSNs includes data extraction, storage and mining techniques, and must be efficient and dependable. In this paper, we propose a new type of behavioral pattern mining technique from sensor data called regularly frequent sensor patterns (RFSPs). RFSPs can identify a set of temporally correlated sensors which can reveal significant knowledge from the monitored data. A distributed data extraction model to prepare the data required for mining RFSPs is proposed, as the distributed scheme ensures higher availability through greater redundancy. The tree structure for RFSP is compact requires less memory and can be constructed using only a single scan through the dataset, and the mining technique is efficient with low runtime. Current mining techniques in the literature on sensor data employ a single memory-based sequential approach and hence are not efficient. Moreover, usage of the. MapReduce model for the distributed solution has not been explored extensively. Since MapReduce is becoming the de facto model for computation on large data, we also propose a parallel implementation of the RFSP mining algorithm, called RFSP on Hadoop (RFSP-H), which uses a MapReduce-based framework to gain further efficiency. Experiments conducted to evaluate the compactness and performance of the data extraction model, RFSP-tree and RFSP-H mining show improved results. (C) 2016 Elsevier Inc. All rights reserved.
Periodic associated sensor patterns mining from wireless sensor networks
- Authors: Rashid, Mamunur , Kamruzzaman, Joarder , Gondal, Iqbal , Hassan, Rafiul
- Date: 2017
- Type: Text , Conference proceedings
- Relation: Proceedings of the 24th International Conference on Neural Information Processing (ICONIP 2017); Guangzhou, China; 14/11/2017-18/11/2017 p. 247-255
- Full Text: false
- Reviewed:
- Description: Mining interesting knowledge from the massive amount of data gathered in wireless sensor networks is a challenging task. Works reported in literature all-confidence measure based associated sensor patterns can captures association-like co-occurrences and the strong temporal correlations implied by such co-occurrences in the sensor data. However, when the user given all-confidence threshold is low, a huge amount of patterns are generated and mining these patterns may not be space and time efficient. Temporal periodicity of pattern appearance can be regarded as an important criterion for measuring the interestingness of associated patterns in WSNs. Associated sensor patterns that occur after regular intervals is called periodic associated sensor patterns. Even though mining periodic associated sensor patterns from sensor data stream is extremely important in many real-time applications, no such algorithm has been proposed yet. In this paper, we propose a compact tree structure called Periodic Associated Sensor Pattern-tree (PASP-tree) and an efficient mining approach for finding periodic associated sensor patterns (PASPs) from WSNs. Extensive performance analyses show that our technique is time and memory efficient in finding periodic associated sensor patterns.
An efficient data extraction framework for mining wireless sensor networks
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2016
- Type: Text , Conference paper
- Relation: 23rd International Conference, ICONIP 2016; Kyoto, Japan; 16th-21st October 2016; published in Neural Information Processing, Part III (Lecture Notes in Computer Science series) Vol. 9949, p. 491-498
- Full Text:
- Reviewed:
- Description: Behavioral patterns for sensors have received a great deal of attention recently due to their usefulness in capturing the temporal relations between sensors in wireless sensor networks. To discover these patterns, we need to collect the behavioral data that represents the sensor's activities over time from the sensor database that attached with a well-equipped central node called sink for further analysis. However, given the limited resources of sensor nodes, an effective data collection method is required for collecting the behavioral data efficiently. In this paper, we introduce a new framework for behavioral patterns called associated-correlated sensor patterns and also propose a MapReduce based new paradigm for extract data from the wireless sensor network by distributed away. Extensive performance study shows that the proposed method is capable to reduce the data size almost 50% compared to the centralized model.
A mapreduce based technique for mining behavioral patterns from sensor data
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2015
- Type: Text , Conference paper
- Relation: 22nd International Conference on Neural Information Processing, ICONIP 2015; Istanbul, Turkey; 9th-12th November 2015 Vol. 9492, p. 145-153
- Full Text: false
- Reviewed:
- Description: WSNs generate a large amount of data in the form of streams, and temporal regularity in occurrence behavior is considered as an important measure for assessing the importance of patterns in WSN data. A frequent sensor pattern that occurs after regular intervals in WSNs is called regularly frequent sensor patterns (RFSPs). Existing RFSPs techniques assume that the data structure of the mining task is small enough to fit in the main memory of a processor. However, given the emergence of the Internet of Things (IoT), WSNs in future will generate huge volume of data, which means such an assumption does not hold any longer. To overcome this, a distributed solution using MapReduce model has not yet been explored extensively. Since MapReduce is becoming the de-facto model for computation on large data, an efficient RFSPs mining algorithm on this model is likely to provide a highly effective solution. In this work, we propose a regularly frequent sensor patterns mining algorithm called RFSP-H which uses MapReduce based framework. Extensive performance analyses show that our technique is significantly time efficient in finding regularly frequent sensor patterns. © Springer International Publishing Switzerland 2015.
Mining associated patterns from wireless sensor networks
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2015
- Type: Text , Journal article
- Relation: IEEE Transactions on Computers Vol. 64, no. 7 (2015), p. 1998-2011
- Full Text: false
- Reviewed:
- Description: Mining of sensor data for useful knowledge extraction is a very challenging task. Existing works generate sensor association rules using occurrence frequency of patterns to extract the knowledge. These techniques often generate huge number of rules, most of which are non-informative or fail to reflect true correlation among sensor data. In this paper, we propose a new type of behavioral pattern called associated sensor patterns which capture association-like co-occurrences as well as temporal correlations which are linked with such co-occurrences. To capture such patterns a compact tree structure, called associated sensor pattern tree (ASP-tree) and a mining algorithm (ASP) are proposed which use pattern growth-based approach to generate all associated patterns with only one scan over dataset. Moreover, when data stream flows through, old information may lose significance for the current time. To capture significance of recent data, ASP-tree is further enhanced to SWASP-tree by adopting sliding observation window and updating the tree structure accordingly. Finally, window size is made dynamically adaptive to ensure efficient resource usage. Different characteristics of the proposed techniques and their computational complexity are presented. Experimental results show that our approach is very efficient in discovering associated sensor patterns and outperforms existing techniques.
Share-frequent sensor patterns mining from wireless sensor network data
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2015
- Type: Text , Journal article
- Relation: IEEE Transactions on Parallel and Distributed Systems Vol. 26, no. 12 (2015), p. 3471-3484
- Full Text: false
- Reviewed:
- Description: Mining interesting knowledge from the huge amount of data gathered from WSNs is a challenge. Works reported in literature use support metric-based sensor association rules which employ the occurrence frequency of patterns as criteria. However, consideration of the binary frequency of a pattern is not a sufficient indicator for finding meaningful patterns because it only reflects the number of epochs which contain that pattern in the dataset. The share measure of sensorsets could discover useful knowledge about trigger values associated with a sensor. Here, we propose a new type of behavioral pattern called share-frequent sensor patterns (SFSPs) by considering the non-binary frequency values of sensors in epochs. SFSPs can find a correlation among a set of sensors and hence can improve the performance of WSNs in a resource management process. In this paper, a share-frequent sensor pattern tree (ShrFSP-Tree) has been proposed to facilitate a pattern growth mining technique to discover SFSPs from WSN data. We also present a parallel and distributed method where the ShrFSP-Tree is enhanced into PShrFSP-Tree and its performance is investigated for both homogeneous and heterogeneous systems. Results show that our method is time and memory efficient in finding SFSPs than the existing most efficient algorithms.
A novel algorithm for mining behavioral patterns from wireless sensor networks
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2014
- Type: Text , Conference paper
- Relation: 2014 International Joint Conference on Neural Networks, IJCNN 2014; Beijing, China; 6th-11th July 2014 p. 1-7
- Full Text: false
- Reviewed:
- Description: Due to recent advances in wireless sensor networks (WSNs) and their ability to generate huge amount of data in the form of streams, knowledge discovery techniques have received a great deal of attention to extract useful knowledge regarding the underlying network. Traditionally sensor association rules measure occurrence frequency of patterns. However, these rules often generate a huge number of rules, most of which are non-informative or fail to reflect the true correlation among data objects. In this paper, we propose a new type of sensor behavioral pattern called associated sensor patterns that captures association-like co-occurrences and the strong temporal correlations implied by such co-occurrences in the sensor data. We also propose a novel tree structure called as associated sensor pattern tree (ASPT) and a mining algorithm, associated sensor pattern (ASP) which facilitates frequent pattern (FP) growth-based technique to generate all associated sensor patterns from WSN data with only one scan over the sensor database. Extensive performance study shows that our algorithm is very efficient in finding associated sensor patterns than the existing significant algorithms.
A technique for parallel share-frequent sensor pattern mining from wireless sensor networks
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2014
- Type: Text , Conference paper
- Relation: 14th Annual International Conference on Computational Science, ICCS 2014; Cairns, Australia; 10th-12th June 2014; published in Procedia Computer Science p. 124-133
- Full Text:
- Reviewed:
- Description: WSNs generate huge amount of data in the form of streams and mining useful knowledge from these streams is a challenging task. Existing works generate sensor association rules using occurrence frequency of patterns with binary frequency (either absent or present) or support of a pattern as a criterion. However, considering the binary frequency or support of a pattern may not be a sufficient indicator for finding meaningful patterns from WSN data because it only reflects the number of epochs in the sensor data which contain that pattern. The share measure of sensorsets could discover useful knowledge about numerical values associated with sensor in a sensor database. Therefore, in this paper, we propose a new type of behavioral pattern called share-frequent sensor patterns by considering the non-binary frequency values of sensors in epochs. To discover share-frequent sensor patterns from sensor dataset, we propose a novel parallel technique. In this technique, we develop a novel tree structure, called parallel share-frequent sensor pattern tree (PShrFSP-tree) that is constructed at each local node independently, by capturing the database contents to generate the candidate patterns using a pattern growth technique with a single scan and then merges the locally generated candidate patterns at the final stage to generate global share-frequent sensor patterns. Comprehensive experimental results show that our proposed model is very efficient for mining share-frequent patterns from WSN data in terms of time and scalability.
ACSP-Tree: A tree structure for mining behavioral patterns from wireless sensor networks
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2013
- Type: Text , Conference paper
- Relation: IEEE Conference on Local Computer Networks (LCN 2013) (21 October 2013 to 24 October 2013) p. 691-694
- Full Text: false
- Reviewed:
- Description: WSNs generates a large amount of data in the form of stream and mining knowledge from the stream of data can be extremely useful. Association rules mining, from the sensor data, has been studied in recent literature. However, sensor association rules mining often produces a huge number of rules, but most of them either are redundant or fail to reflect the true correlation relationship among data objects. In this paper, we address this problem and propose mining of a new type of sensor behavioral pattern called associated-correlated sensor patterns. The proposed behavioral patterns capture not only association-like co-occurrences but also the substantial temporal correlations implied by such co-occurrences in the sensor data. Here, we also use a prefix tree-based structure called associated-correlated sensor pattern-tree (ACSP-tree), which facilitates frequent pattern (FP) growth-based mining technique to generate all associated-correlated patterns from WSN data with only one scan over the sensor database. Extensive performance study shows that our approach is time and memory efficient in finding associated-correlated patterns than the existing most efficient algorithms.
Mining associated sensor patterns for data stream of wireless sensor networks
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2013
- Type: Text , Conference proceedings
- Relation: 8th ACM International Workshop on Performance Monitoring, Measurement, and Evaluation of Heterogeneous Wireless and Wired Networks, PM2HW2N 2013, Barcelona; Spain; 3rd-8th November 2013 p. 91-98
- Full Text: false
- Reviewed:
- Description: WSNs generate a large amount of data in the form of data stream; and mining these streams to extract useful knowledge is a highly challenging task. Existing works proposed in literature use sensor association rules measured in terms of occurrence frequency of patterns. However, these rules often generate a huge number of rules, most of which are non-informative or fail to reflect the true correlation among data objects. Additionally mining associated sensor patterns from sensor stream data, which is vital for real-time applications, has not been addressed yet in literature. In this paper, we address these problems and propose a new type of sensor behavioral pattern called associated sensor patterns which capture simultaneously association-like co-occurrence as well as substantial temporal correlations implied by such co-occurrences in sensor data. We propose a novel tree structure, called associated sensor pattern stream tree (ASPS-tree) and a new technique, called associated sensor pattern mining of data stream (ASPMS), using sliding window-based associated sensor pattern mining for WSNs. By capturing the useful knowledge of the data stream into an ASPS-tree, our ASPMS algorithm can mine associated sensor patterns in the current window with frequent pattern (FP)-growth like pattern-growth method. Extensive experimental analyses show that our technique is very efficient in discovering associated sensor patterns over sensor data stream.
Regularly frequent patterns mining from sensor data stream
- Authors: Rashid, Md. Mamunur , Gondal, Iqbal , Kamruzzaman, Joarder
- Date: 2013
- Type: Text , Conference paper
- Relation: International Conference on Neural Information Processing (ICONIP 2013) p. 417-424
- Full Text: false
- Reviewed:
- Description: Mining interesting and useful knowledge from the huge amount of data gathered in wireless sensor networks is a challenging task. Works reported in literature use support metric-based sensor association rule which employs the occurrence frequency of patterns as criteria. Such criteria may not be appropriate for finding significant patterns. Moreover, temporal regularity in occurrence behavior should be considered as another important measure for assessing the importance of patterns in WSNs. Frequent sensor patterns that occur after regular intervals is called regularly frequent sensor patterns. Even though mining regularly frequent sensor patterns from sensor data stream is extremely important in many real-time applications, no such algorithm has been proposed yet. In this paper, we propose a novel tree structure called Regularly Frequent Sensor Pattern-tree (RSP-tree) and an efficient mining approach for finding regularly frequent sensor patterns from WSNs. Extensive performance analyses show that our technique is time and memory efficient in finding regularly frequent sensor patterns.