Improving Identity Consistency for Neural Head Reeanactment

Document Type

Dissertation

Abstract

"Neural reenactment systems are becoming increasingly popular for generating realistic human portraits by extracting the pose and expression of a driver image and transferring it to a source image. However, current solutions struggle with identity change and leakage, and can result in inconsistent and undesirable facial expressions. In this thesis, we propose an approach that disentangles driver and source identities to improve identity consistency while maintaining high-fidelity results. Our approach utilizes multi-frame support, sparsity in expression latent vectors, and high-quality datasets for training. We also introduce a super-resolution module to further enhance our results. Our contributions include a neural reenactment model that improves identity consistency, techniques for encouraging disentanglement, curation of high-quality reenactment datasets, and a super-resolution module."

First Page

i

Last Page

29

Publication Date

6-2023

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfillment of the requirements for the M.Sc degree in Machine Learning

Advisors: Dr. Hao Li, Dr. Martin Takac

Online access available for MBZUAI patrons

Link to Full Text

Share

COinS