Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild

Document Type

Article

Publication Title

International Journal of Computer Vision

Abstract

Recently, heatmap regression models have become popular due to their superior performance in locating facial landmarks. However, three major problems still exist among these models: (1) they are computationally expensive; (2) they usually lack explicit constraints on global shapes; (3) domain gaps are commonly present. To address these problems, we propose Pixel-in-Pixel Net (PIPNet) for facial landmark detection. The proposed model is equipped with a novel detection head based on heatmap regression, which conducts score and offset predictions simultaneously on low-resolution feature maps. By doing so, repeated upsampling layers are no longer necessary, enabling the inference time to be largely reduced without sacrificing model accuracy. Besides, a simple but effective neighbor regression module is proposed to enforce local constraints by fusing predictions from neighboring landmarks, which enhances the robustness of the new detection head. To further improve the cross-domain generalization capability of PIPNet, we propose self-training with curriculum. This training strategy is able to mine more reliable pseudo-labels from unlabeled data across domains by starting with an easier task, then gradually increasing the difficulty to provide more precise labels. Extensive experiments demonstrate the superiority of PIPNet, which obtains new state-of-the-art results on three out of six popular benchmarks under the supervised setting. The results on two cross-domain test sets are also consistently improved compared to the baselines. Notably, our lightweight version of PIPNet runs at 35.7 FPS and 200 FPS on CPU and GPU, respectively, while still maintaining a competitive accuracy to state-of-the-art methods. The code of PIPNet is available at https://github.com/jhb86253817/PIPNet.

First Page

3174

Last Page

3194

DOI

10.1007/s11263-021-01521-4

Publication Date

9-16-2021

Keywords

Facial landmark detection, Pixel-in-pixel regression, Self-training with curriculum, Unsupervised domain adaptation

Comments

IR Deposit conditions:

OA version (pathway b) Accepted version

12 month embargo

Publisher's Bespoke License

Published source must be acknowledged with citation

Must link to publisher version with DOI

Post-prints are subject to Springer Nature re-use terms

Set statement to accompany deposit (see policy)

Link to Full Text

Share

COinS