Fast Anomaly Detection for IoT Services Based on Multisource Log Fusion

Document Type

Article

Publication Title

IEEE Internet of Things Journal

Abstract

With the fast development of Internet of Things (IoT), anomaly detection has recently become a common concern for IoT smart applications. The circuit detection mode of all services to detect application anomaly is widely adopted. However, regularly collecting key performance indicators (KPIs) of all services under different clouds is a time-consuming task. Moreover, too many alerts in a short period of time will affect the processing speed of engineers. Recently, some researches of inferring the key service on the call paths can solve the above problems. However, it is still a challenge to locate dynamic and scattered key services accurately. In this paper, we propose a service inference-based anomaly detection approach (SIADA), which integrates three sources of logs: call, business and metric. SIADA leverages the deep graph representation with context-aware multi-graph fusion based on a recurrent encoder. This should to infer key services, and adopts Variational Auto-Encoder (VAE) with the flow model to detect multivariate time series anomalies for key services. We have conducted extensive experiments on the public dataset MicroSS. SIADA achieves the best average accuracy of 92% in service inference and best average F1-score of 0.98 in anomaly detection, which is improved by 12.17% and 6.42% compared with the best SOTA baseline, respectively. Moreover, the total detection time, network transmission and average alert number are reduced by 42.12%, 81.87%, and 83.5%, respectively.

First Page

9405

Last Page

9419

DOI

10.1109/JIOT.2023.3323620

Publication Date

10-11-2023

Keywords

Anomaly detection, Cloud computing, Data models, Fast anomaly detection, Internet of Things, IoT service, Key service inference, Log fusion, Monitoring, Service operation, Time series analysis, Topology

Comments

IR conditions: non-described

Link to Full Text

Share

COinS