A Practical Multi-View System for Building Neural Head Avatars

Date of Award

4-30-2024

Document Type

Thesis

Degree Name

Master of Science in Computer Vision

Department

Computer Vision

First Advisor

Dr. Hao Li

Second Advisor

Dr. Preslav Nakov

Abstract

Digital human head avatars show promise in applications, including the expanding metaverse, digital telepresence, entertainment, and educational tools. However, the traditional methods, such as manual modelling and photogrammetry, to create such models are time-consuming, laborious or prone to human error. Current automated processes such as Light Stage are prohibitively expensive and require highly specialised engineering to build, so they are not widely available. This work explores the feasibility of deep-learning neural methods to represent four-dimensional head avatars captured in 360 degrees with the addition of dynamic movements, allowing for the digital recreation of any real human head given only sparse calibrated images. An end-to-end pipeline is proposed and executed from hardware set-up to avatar rendering, including the creation of a sparse-view dataset using a non-rigid system such that camera positions and orientations can be adjusted for testing different configurations. It is found that currently available state-of-the-art methods focusing on frontal facial capture using multi-resolution spatial hashing representations may be extended to a 360-degree sparse setting while retaining reasonable quality, suggesting directions for the future development of head avatar capturing systems.

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfilment of the requirements for the M.Sc degree in Computer Vision

Advisors: Hao Li, Preslav Nakov

Online access available for MBZUAI patrons

Link to Full Text

Share

COinS