Modal-aware Resource Allocation for Cross-modal Collaborative Communication in IIoT

Document Type


Publication Title

IEEE Internet of Things Journal


With the development of human-machine interactions, users are increasingly evolving towards an immersion experience with multi-dimensional stimuli. Facing this trend, cross-modal collaborative communication is considered an effective technology in the Industrial Internet of Things (IIoT). In this paper, we focus on open issues about resource reuse, pair interactivity, and user assurance in cross-modal collaborative communication to improve quality of service (QoS) and users’ satisfaction. Therefore, we propose a novel architecture of modal-aware resource allocation to solve these contradictions. First, taking all the characteristics of multi-modal into account, we introduce network slices to visualize resource allocation, which is modeled as a Markov Decision Process (MDP). Second, we decompose the problem by the transformation of probabilistic constraint and Lyapunov Optimization. Third, we propose a deep reinforcement learning (DRL) decentralized method in the dynamic environment. Meanwhile, a federated DRL framework is provided to overcome the training limitations of local DRL models. Finally, numerical results demonstrate that our proposed method performs better than other decentralized methods and achieves superiority in cross-modal collaborative communications.

First Page


Last Page




Publication Date



Collaboration, cross-modal collaborative communication, DRL, federated learning, IIoT, Industrial Internet of Things, Markov processes, MDP, Reliability, resource allocation, Resource management, Streaming media, Ultra reliable low latency communication


IR Deposit conditions:

OA version (pathway a) Accepted version

No embargo

When accepted for publication, set statement to accompany deposit (see policy)

Must link to publisher version with DOI

Publisher copyright and source must be acknowledged