- Title
- Categorical features transformation with compact one-hot encoder for fraud detection in distributed environment
- Creator
- Ul Haq, Ikram; Gondal, Iqbal; Vamplew, Peter; Brown, Simon
- Date
- 2019
- Type
- Text; Conference proceedings; Conference paper
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/180898
- Identifier
- vital:15847
- Identifier
-
https://doi.org/10.1007/978-981-13-6661-1_6
- Identifier
- ISBN:18650929 (ISSN); 9789811366604 (ISBN)
- Abstract
- Fraud detection for online banking is an important research area, but one of the challenges is the heterogeneous nature of transactions data i.e. a combination of numeric as well as mixed attributes. Usually, numeric format data gives better performance for classification, regression and clustering algorithms. However, many machine learning problems have categorical, or nominal features, rather than numeric features only. In addition, some machine learning platforms such as Apache Spark accept numeric data only. One-hot Encoding (OHE) is a widely used approach for transforming categorical features to numerical features in traditional data mining tasks. The one-hot approach has some challenges as well: the sparseness of the transformed data and that the distinct values of an attribute are not always known in advance. Other than the model accuracy, compactness of machine learning models is equally important due to growing memory and storage needs. This paper presents an innovative technique to transform categorical features to numeric features by compacting sparse data even if all the distinct values are not known. The transformed data can be used for the development of fraud detection systems. The accuracy of the results has been validated on synthetic and real bank fraud data and a publicly available anomaly detection (KDD-99) dataset on a multi-node data cluster. © Springer Nature Singapore Pte Ltd. 2019.
- Publisher
- Springer Verlag
- Relation
- 2019 16th Australasian Conference on Data Mining, AusDM 2018; Bathurst, NSW; 28 November 2018 through 30 November 2018 Vol. 996, p. 69-80
- Rights
- All metadata describing materials held in, or linked to, the repository is freely available under a CC0 licence
- Rights
- Copyright Springer
- Subject
- Categorical data; Compactness; Distributed computing; Hadoop; HDFS; Machine learning; One-hot encoder; Spark; Sparse data; Anomaly detection; Clustering algorithms; Crime; Data mining; Digital storage; Distributed computer systems; Electric sparks; Learning systems; Signal encoding; State assignment; Metadata
- Reviewed
- Hits: 3601
- Visitors: 3190
- Downloads: 1
| Thumbnail | File | Description | Size | Format |
|---|