Towards a Digital Twin of Cognitive Processes

Document Type

Dissertation

Abstract

In recent years, the emerging field of digital twins has witnessed rapid developments due to the advancement in industry 4.0 technologies such as robotics, the internet of things (IoT) and artificial intelligence (AI). Formally introduced within the context of product lifecycle management, the digital twin framework has been implemented in various areas, prominently within manufacturing and engineering. Nevertheless, there still lies an opportunity to explore its full potential in health and well-being. In particular, the field of cognitive health could benefit from a personalized approach to long-term patient care. This thesis proposes a reproducible framework integrating the latest deep learning models with digital twin concepts to address an electroencephalography (EEG) signal classification problem. Our methodology is four-dimensional and mainly considers cross-subject analysis. First, we extend a simple convolutional neural network (CNN) architecture to integrate an individualized model that learns a latent representation of the subject’s experience. This addition enables the development of a digital twin associated with each subject that learns over time and can be utilized in various ways, such as analysis of cognitive psychology, lifestyle habits, or a person’s most productive learning style. Second, we build upon the first approach to incorporate a second digital twin model, which follows the downstream task and captures how the individual actually perceived the cognitive experience. Third, we apply our framework to a more complex multi-scale CNN to demonstrate its effectiveness and reproducibility. Lastly, to account for realistic problem settings with limited privacy-sensitive datasets, we investigate the feasibility of our method within a personalized federated learning setting. We establish the efficacy of our proof of concept in tackling raw EEG signal classification using four datasets related to three cognitive tasks: motor imagery, motor movement, and event-related potential. The first dataset includes nine subjects performing a 4-class motor imagery task. In contrast, the second dataset captures the event-related potential associated with 29 subjects undergoing a 3-class haptic experience. On the other hand, the third and fourth datasets consist of 109 subjects performing a 2-class motor imagery and motor movement task, respectively. Besides incorporating personalization, our approach significantly improves test accuracy on the baseline in most aforementioned settings. For instance, incorporating one digital twin model to a simple CNN baseline resulted in a 6.42 % and 2.31 % improvement in average cross-subject test accuracy for the 9-subject motor imagery and 29-subject event-related potential datasets, respectively. Whereas introducing two digital twin models improved the average test accuracy on the same datasets by 11.27% and 4.75%, respectively. Moreover, extending a more complex CNN baseline to incorporate a digital twin did not compromise general disability for the 9-subject and 109-subject motor imagery tasks, resulting in a marginal improvement in average test accuracy. Finally, this work demonstrates the potential behind leveraging the digital twin framework for deep learning classification problems that address personal data. Promising research directions include developing a federated digital twin framework and exploring its application in online shopping and social media platforms.

First Page

i

Last Page

54

Publication Date

12-30-2022

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfillment of the requirements for the M.Sc degree in Machine Learning

Advisors: Dr. Mohammad Yaqub, Dr. Le Song

Online access provided for MBZUAI patrons

Link to Full Text

Share

COinS