A Virtual Try-On System for Any Clothing and Body Model

Date of Award

4-30-2024

Document Type

Thesis

Degree Name

Master of Science in Computer Vision

Department

Computer Vision

First Advisor

Dr. Hao Li

Second Advisor

Dr. Kun Zhang

Abstract

Virtual try-on (VTON) technology is a powerful tool for e-commerce, fashion design, and digital content creation. However, previous work, like physical-based simulation, although it provides realistic results, requires expertise and manual adjustment and can not achieve real time performance. In this study, we propose to develop a deep learning-based cloth simulation framework that can quickly fit any clothing model onto any body model, regardless of whether the body model is parametric or non-parametric. Our system not only automates the previously labor-intensive process of fitting garments to varied body shapes and poses, but also introduces flexibility and customization capabilities in garment sizing and styling. Our system is composed of registration module, cloth transfer module, and collisionsolving module. Firstly, we render the original body model in multiple views, detect keypoints and joints from the 2D images, and lift 2D pose to 3D as a nice initialization. Thus, the SMPL with the initialized pose is optimized until the shape and pose of the SMPL align with the original model. Secondly, the network integrates the loss functions that formed from physical principles, thus governing cloth movement and deformation in a realistic way and realizing self-supervised training. The inference pipeline is implemented to realize smooth garment mesh adaptation across diverse body types. Thirdly, the collision solving module is used to produce a collision-free result. The experiment results prove the system’s effectiveness, showcasing its ability to accurately dress clothing on different body types and efficiently manage challenging situations like collisions and self-collisions. The enhanced style customization brings more possibilities for creative users, makes our work a reliable and adaptable tool for the fashion industry and other fields.

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfilment of the requirements for the M.Sc degree in Computer Vision

Advisors: Hao Li, Kun Zhang

with 1 year embargo period

This document is currently not available here.

Share

COinS