- Title
- Progressive data stream mining and transaction classification for workload-aware incremental database repartitioning
- Creator
- Kamal, Joarder; Murshed, Manzur; Gaber, Mohamed
- Date
- 2014
- Type
- Text; Conference proceedings
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/155206
- Identifier
- vital:11266
- Identifier
-
https://doi.org/10.1109/BDC.2014.8
- Identifier
- ISBN:978-1-47991897-3
- Abstract
- Minimising the impact of distributed transactions (DTs) in a shared-nothing distributed database is extremely challenging for transactional workloads. With dynamic workload nature and rapid growth in data volume the underlying database requires incremental repartitioning to maintain acceptable level of DTs and data load balance with minimum physical data migrations. In a workload-aware repartitioning scheme transactional workload is modelled as graph or hyper graph, and subsequently perform k-way min-cut clustering guaranteeing minimum edge cuts can reduce the impact of DTs significantly by mapping the workload clusters into logical database partitions. However, without exploring the inherent workload characteristics, the overall processing and computing times for large-scale workload networks increase in polynomial orders. In this paper, a workload-aware incremental database repartitioning technique is proposed, which effectively exploits proactive transaction classification and workload stream mining techniques. Workload batches are modelled in graph, hyper graph, and compressed hyper graph then repartitioned to produce a fresh tuple-to-partition data migration plan for every incremental cycle. Experimental studies in a simulated TPC-C environment demonstrate that the proposed model can be effectively adopted in managing rapid data growth and dynamic workloads, thus progressively reduce the overall processing time required to operate over the workload networks.
- Publisher
- IEEE
- Relation
- IEEE/ACM International Symposium on Big Data Computing, BDC 2014; London, United Kingdom; 8th-11th December 2014; p. 8-15
- Rights
- Copyright © 2014 by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved.
- Rights
- Open Access
- Rights
- This metadata is freely available under a CCO license
- Subject
- Cloud Databases; Workload; Distributed Transactions; Data Stream Mining; Classification; Incremental Repartitioning; Load-Balance; Data Migration
- Full Text
- Reviewed
- Hits: 1316
- Visitors: 1781
- Downloads: 529
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Accepted version | 592 KB | Adobe Acrobat PDF | View Details Download | ||
| View Details Download | SOURCE2 | Published Version | 354 KB | Adobe Acrobat PDF | View Details Download |