Pre-trained language models with limited data for intent classification
- Kasthuriarachchy, Buddhika, Chetty, Madhu, Karmakar, Gour, Walls, Darren
- Authors: Kasthuriarachchy, Buddhika , Chetty, Madhu , Karmakar, Gour , Walls, Darren
- Date: 2020
- Type: Text , Conference proceedings , Conference paper
- Relation: 2020 International Joint Conference on Neural Networks, IJCNN 2020
- Full Text: false
- Reviewed:
- Description: Intent analysis is capturing the attention of both the industry and academia due to its commercial and noncommercial significance. The rapid growth of unstructured data of micro-blogging platforms, such as Twitter and Facebook, are amongst the important sources for intent analysis. However, the social media data are often noisy and diverse, thus making the task very challenging. Further, the intent analysis frequently suffers from lack of sufficient data because the labeled datasets are often manually annotated. Recently, BERT (Bidirectional Encoder Representation from Transformers), a state-of-the-art language representation model, has attracted attention for accurate language modelling. In this paper, we investigate the application of BERT for its suitability for intent analysis. We study the fine-tuning of the BERT model through inductive transfer learning and investigate methods to overcome the challenges due to limited data availability by proposing a novel semantic data augmentation approach. This technique generates synthetic sentences while preserving the label-compatibility using the semantic meaning of the sentences, to improve the intent classification accuracy. Thus, based on the considerations for finetuning and data augmentation, a systematic and novel step-bystep methodology is presented for applying the linguistic model BERT for intent classification with limited data available. Our results show that the pre-trained language can be effectively used with noisy social media data to achieve state-of-the-art accuracy in intent analysis under low labeled-data regime. Moreover, our results also confirm that the proposed text augmentation technique is effective in eliminating noisy synthetic sentences, thereby achieving further performance improvements. © 2020 IEEE.
Assessing transformer oil quality using deep convolutional networks
- Alam, Mohammad, Karmakar, Gour, Islam, Syed, Kamruzzaman, Joarder, Chetty, Madhu, Lim, Suryani, Appuhamillage, Gayan, Chattopadhyay, Gopi, Wilcox, Steve, Verheyen, Vincent
- Authors: Alam, Mohammad , Karmakar, Gour , Islam, Syed , Kamruzzaman, Joarder , Chetty, Madhu , Lim, Suryani , Appuhamillage, Gayan , Chattopadhyay, Gopi , Wilcox, Steve , Verheyen, Vincent
- Date: 2019
- Type: Text , Conference proceedings , Conference paper
- Relation: 29th Australasian Universities Power Engineering Conference, AUPEC 2019
- Full Text:
- Reviewed:
- Description: Electrical power grids comprise a significantly large number of transformers that interconnect power generation, transmission and distribution. These transformers having different MVA ratings are critical assets that require proper maintenance to provide long and uninterrupted electrical service. The mineral oil, an essential component of any transformer, not only provides cooling but also acts as an insulating medium within the transformer. The quality and the key dissolved properties of insulating mineral oil for the transformer are critical with its proper and reliable operation. However, traditional chemical diagnostic methods are expensive and time-consuming. A transformer oil image analysis approach, based on the entropy value of oil, which is inexpensive, effective and quick. However, the inability of entropy to estimate the vital transformer oil properties such as equivalent age, Neutralization Number (NN), dissipation factor (tanδ) and power factor (PF); and many intuitively derived constants usage limit its estimation accuracy. To address this issue, in this paper, we introduce an innovative transformer oil analysis using two deep convolutional learning techniques such as Convolutional Neural Network (ConvNet) and Residual Neural Network (ResNet). These two deep neural networks are chosen for this project as they have superior performance in computer vision. After estimating the equivalent aging year of transformer oil from its image by our proposed method, NN, tanδ and PF are computed using that estimated age. Our deep learning based techniques can accurately predict the transformer oil equivalent age, leading to calculate NN, tanδ and PF more accurately. The root means square error of estimated equivalent age produced by entropy, ConvNet and ResNet based methods are 0.718, 0.122 and 0.065, respectively. ConvNet and ResNet based methods have reduced the error of the oil age estimation by 83% and 91%, respectively compared to that of the entropy method. Our proposed oil image analysis can calculate the equivalent age that is very close to the actual age for all images used in the experiment. © 2019 IEEE.
- Description: E1
- Authors: Alam, Mohammad , Karmakar, Gour , Islam, Syed , Kamruzzaman, Joarder , Chetty, Madhu , Lim, Suryani , Appuhamillage, Gayan , Chattopadhyay, Gopi , Wilcox, Steve , Verheyen, Vincent
- Date: 2019
- Type: Text , Conference proceedings , Conference paper
- Relation: 29th Australasian Universities Power Engineering Conference, AUPEC 2019
- Full Text:
- Reviewed:
- Description: Electrical power grids comprise a significantly large number of transformers that interconnect power generation, transmission and distribution. These transformers having different MVA ratings are critical assets that require proper maintenance to provide long and uninterrupted electrical service. The mineral oil, an essential component of any transformer, not only provides cooling but also acts as an insulating medium within the transformer. The quality and the key dissolved properties of insulating mineral oil for the transformer are critical with its proper and reliable operation. However, traditional chemical diagnostic methods are expensive and time-consuming. A transformer oil image analysis approach, based on the entropy value of oil, which is inexpensive, effective and quick. However, the inability of entropy to estimate the vital transformer oil properties such as equivalent age, Neutralization Number (NN), dissipation factor (tanδ) and power factor (PF); and many intuitively derived constants usage limit its estimation accuracy. To address this issue, in this paper, we introduce an innovative transformer oil analysis using two deep convolutional learning techniques such as Convolutional Neural Network (ConvNet) and Residual Neural Network (ResNet). These two deep neural networks are chosen for this project as they have superior performance in computer vision. After estimating the equivalent aging year of transformer oil from its image by our proposed method, NN, tanδ and PF are computed using that estimated age. Our deep learning based techniques can accurately predict the transformer oil equivalent age, leading to calculate NN, tanδ and PF more accurately. The root means square error of estimated equivalent age produced by entropy, ConvNet and ResNet based methods are 0.718, 0.122 and 0.065, respectively. ConvNet and ResNet based methods have reduced the error of the oil age estimation by 83% and 91%, respectively compared to that of the entropy method. Our proposed oil image analysis can calculate the equivalent age that is very close to the actual age for all images used in the experiment. © 2019 IEEE.
- Description: E1
- «
- ‹
- 1
- ›
- »