Open-World 3D instance segmentation

Date of Award

4-30-2024

Document Type

Thesis

Degree Name

Master of Science in Computer Vision

Department

Computer Vision

First Advisor

Prof. Hao Li

Second Advisor

Prof. Abdulmotaleb Elsaddik

Abstract

Existing methods for 3D instance segmentation often operate under the closed-world assumption, where only seen categories are segmented at inference, limiting their adaptability to real-world scenarios. In this work, we pioneer open-world 3D indoor instance segmentation, enabling the model to distinguish between known and unknown classes while incrementally learning the semantic category of unknown objects. Our approach employs an auto-labeling scheme during training to generate pseudo-labels, enhancing separation between known and unknown categories. We refine pseudo-label quality at inference by adjusting unknown class probabilities based on objectness score distributions. Furthermore, we introduce curated open-world splits, reflecting realistic scenarios, to evaluate our method comprehensively. Moreover, we propose an exemplar-free approach that integrates continual learning and unknown class identification, leveraging self-distillation. By utilizing pseudo-labels from previous tasks, our method improves unknown predictions during training and mitigates catastrophic forgetting. This unified approach outperforms traditional methods, showcasing superior performance in continual learning and unknown class retrieval. Extensive experiments on various ScanNet200 dataset splits validate the efficacy of our proposed approach, demonstrating its potential for real-world applications.

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfilment of the requirements for the M.Sc degree in Computer Vision

Advisors: Fahad Khan, Merouane Debbah

with 2 years embargo period

This document is currently not available here.

Share

COinS