Contextual Dependency Vision Transformer for spectrogram-based multivariate time series analysis

Authors

Jieru Yao, Northwestern Polytechnical University
Longfei Han, Beijing Technology and Business University
Kaihui Yang, Nanchang University
Guangyu Guo, Northwestern Polytechnical University
Nian Liu, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Xiankai Huang, Beijing Technology and Business University
Zhaohui Zheng, Xijing Hospital
Dingwen Zhang, Northwestern Polytechnical University
Junwei Han, Northwestern Polytechnical University

Document Type

Article

Publication Title

Neurocomputing

Abstract

Multivariate time series (MTS) analysis plays an important role in various real-world applications. Existing Transformer-based methods address this problem based on hierarchical semantic representations across different scales. However, most of them ignore exploiting the helpful multiple temporal and variable relationships within the hierarchical semantic representations. To this end, this paper proposes a novel method named Contextual Dependency Vision Transformer (CD-ViT), which generates multi-grained semantic information based on spectrogram and explores mutual dependencies between multi-variable and multi-temporal representations. CD-ViT contains two key modules, i.e., the Hierarchical Variable-dependency Transformer (HVT) module and the Bidirectional Temporal-dependency Interaction (BTI) module. Specifically, the HVT module progressively establishes mutual dependencies between multiple variables, from fine to coarse scales, with shared parameters. The BTI module employs two bidirectional flows to fuse multi-temporal tokens through zoom-in and zoom-out operations. Comprehensive experiments on widely used datasets, including UEA, Olszewski, UCI, MIMIC III, and ETT, demonstrate that the proposed approach achieves significant improvement on three popular tasks, i.e., classification, regression, and forecasting. The code is available at https://github.com/Kali-github/CD-ViT.

DOI

10.1016/j.neucom.2023.127215

Publication Date

3-1-2024

Keywords

Hierarchical vision-style transformer, Multi-temporal dependency, Multi-variables dependency, Multivariate times series analysis, Spectrogram-based contextual interaction, Harmonic analysis, Multivariant analysis, Semantics, Spectrographs, Time series analysis

Comments

IR Deposit conditions:

OA version (pathway b) Accepted version

24 month embargo

License: CC BY-NC-ND

Must link to publisher version with DOI

Recommended Citation

J. Yao et al., "Contextual Dependency Vision Transformer for spectrogram-based multivariate time series analysis," Neurocomputing, vol. 572, Mar 2024.

The definitive version is available at https://doi.org/10.1016/j.neucom.2023.127215

Additional Links

https://doi.org/10.1016/j.neucom.2023.127215