Data-efficient transformer-based 3D object detection

Document Type

Dissertation

Abstract

This thesis work intends to study 3D point clouds object detection from indoor scenes. 3D object detection aims to recognize classes of object categories and locate them by drawing a bounding box around an object. To achieve this final goal of predicting a set of bounding boxes from an input scan, a machine learning model must extract necessary features that describe the scene points. Nevertheless, at this moment, 3D scene under- standing poses a challenge, as 3D point cloud data is unique: it is orderless, sparse, and continuous. Recent 3D detection models rely on Transformer architecture due to its natural ability to abstract global context features. One is the 3DETR network - a pure transformer-based model designed to generate 3D boxes on indoor dataset scans. It is generally known that transformers are data-hungry. However, data collection and annotation in 3D are more challenging than in 2D. Thus, our goal is to study the data-hungriness of the 3DETR-m model and propose a solution for its data efficiency. Our methodology is based on the observation that PointNet++ provides more locally aggregated features that can be useful to support 3DETR-m prediction on small dataset problem. We suggest three methods of backbone fusion that are based on addition (Fusion I), concatenation (Fusion II), and replacement (Fusion III). We utilize pre-trained weights from the Group-free model trained on the SUN RGB-D dataset. The proposed 3DETR-m outperforms the original model in all data proportions (10%, 25%, 50%, 75%, and 100%). We improve 3DETR-m paper results by 1.46% and 2.46% in mAP@25 and mAP@50 on the full dataset. Hence, we believe our research efforts can provide new insights into the data- hungriness issue of 3D transformer detectors and inspire the usage of pre-trained models in 3D as one way towards data efficiency.

First Page

i

Last Page

63

Publication Date

12-1-2022

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfillment of the requirements for the M.Sc degree in Computer Vision

Advisors: Dr. Hisham Cholakkal, Dr. Fahad Khan

with permanent embargo

This document is currently not available here.

Share

COinS