- Title
- In your face : sentiment analysis of metaphor with facial expressive features
- Creator
- Zhang, Dongyu; Zhang, Minghao; Guo, Teng; Peng, Ciyuan; Saikrishna, Vidya; Xia, Feng
- Date
- 2021
- Type
- Text; Conference paper
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/180340
- Identifier
- vital:15753
- Identifier
-
https://doi.org/10.1109/IJCNN52387.2021.9533972
- Identifier
- ISBN:9780738133669 (ISBN)
- Abstract
- Metaphor plays an important role in human communication, which often conveys and evokes sentiments. Numerous approaches to sentiment analysis of metaphors have thus gained attention in natural language processing (NLP). The primary focus of these approaches is on linguistic features and text rather than other modal information and data. However, visual features such as facial expressions also play an important role in expressing sentiments. In this paper, we present a novel neural network approach to sentiment analysis of metaphorical expressions that combines both linguistic and visual features and refer to it as the multimodal model approach. For this, we create a Chinese dataset, containing textual data from metaphorical sentences along with visual data on synchronized facial images. The experimental results indicate that our multimodal model outperforms several other linguistic and visual models, and also outperforms the state-of-the-art methods. The contribution is realized in terms of novelty of the approach and creation of a new, sizeable, and scarce dataset with linguistic and synchronized facial expressive image data. The dataset is particularly useful in languages other than English and the approach addresses one of the most challenging NLP issue: sentiment analysis in metaphor. © 2021 IEEE.
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Relation
- 2021 International Joint Conference on Neural Networks, IJCNN 2021 Vol. 2021-July
- Rights
- All metadata describing materials held in, or linked to, the repository is freely available under a CC0 licence
- Rights
- Copyright © 2021 IEEE
- Rights
- Open Access
- Subject
- Facial Images; Linguistics Features; Metaphor Identification; Multimodal Model; Sentiment Analysis
- Full Text
- Reviewed
- Hits: 749
- Visitors: 762
- Downloads: 61
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Accepted version | 922 KB | Adobe Acrobat PDF | View Details Download |