A Multimodal Coupled Graph Attention Network for Joint Traffic Event Detection and Sentiment Classification
Document Type
Article
Publication Title
IEEE Transactions on Intelligent Transportation Systems
Abstract
Traffic events are one of the main causes of traffic accidents, leading to traffic event detection being a challenging research problem in traffic management and intelligent transportation systems (ITSs). The main gap in this task lies in how to extract and represent the valuable information from various kinds of traffic data. Considering the important role that social networks play in traffic data analysis, we argue that sentiment classification and traffic event detection are two closely related tasks in ITSs, where event and sentiment can reveal both explicit and implicit traffic accidents, respectively. Unfortunately, none of the recent approaches in traffic event detection have taken sentiment knowledge into view. This paper proposes a multimodal coupled graph attention network (MCGAT). It aims to construct a multimodal multitask interactive graphical structure where terms (sucha as words, and pixels) are treated as nodes, and their contextual and cross-modal correlations are formalized as edges. The key components are cross-modal and cross-task graph connection layers. The cross-modal graph connection layer captures the multimodal representation, where each node in one modality connects all nodes in another modality. The cross-task graph connection layer is designed by connecting the multimodal node in one task to two single nodes in another task. Empirical evaluation of two benchmarking datasets, such as MGTES and Twitter, shows the effectiveness of the proposed model over state-of-the-art baselines in terms of F1 and accuracy, with significant improvements of 2.4%, 2.4%, 2.7%, and 2.7%.
First Page
8542
Last Page
8554
DOI
10.1109/TITS.2022.3205477
Publication Date
8-1-2023
Keywords
graph embedding, graph neural networks, intelligent transportation system, sentiment classification, Traffic event detection
Recommended Citation
Y. Zhang, P. Tiwari, Q. Zheng, A. E. Saddik and M. S. Hossain, "A Multimodal Coupled Graph Attention Network for Joint Traffic Event Detection and Sentiment Classification," in IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 8, pp. 8542-8554, Aug. 2023, doi: 10.1109/TITS.2022.3205477.
Comments
IR Deposit conditions:
OA version (pathway a) Accepted version
No embargo
When accepted for publication, set statement to accompany deposit (see policy)
Must link to publisher version with DOI
Publisher copyright and source must be acknowledged