Auto-Labelled Prompting for Remote Sensing Scene Classification

Date of Award

4-30-2024

Document Type

Thesis

Degree Name

Master of Science in Machine Learning

Department

Machine Learning

First Advisor

Dr. Hisham Cholakkal

Second Advisor

Dr. Fahad Khan

Abstract

With the rise of large-scale pre-trained Vision-Language Models (VLMs), there’s been a surge in interest in specific domains such as remote sensing, leading to the development of domain pre-trained VLMs like GeoRSCLIP. However, achieving robust model performance in remote sensing image scene classification without extensive labeling efforts remains a significant challenge. Current adaptation methods that lack labeled supervision typically involve full fine-tuning of the model. In this work, we introduce a label-free, lightweight adaptation approach called auto-labelled prompt tuning for remote sensing (ALP-RS). Our method capitalizes on the extensive contextual knowledge embedded within Large Language Models (LLMs) to construct a class description-based text embedding (CDTE) classifier. Leveraging this enriched CDTE classifier, we generate pseudo labels that align a pre-trained Semi-Supervised Learning (SSL) encoder, referred to as a text-aligned autolabel (TAAL) network, with the visual features within the VLM embedding space. The TAAL network, when aligned in the VLM space, serves as an auto-labeler for guiding the adaptation of the VLM vision encoder through prompt tuning. Our approach eliminates the necessity for manual choices during training, addressing concerns related to memory consumption for downstream adaptation and issues of overfitting. Through our experimental evaluation, we illustrate the effectiveness of our proposed method as it outperforms existing techniques, including those employing full fine-tuning, in terms of Top-1 accuracy. Furthermore, our method maintains competitive performance across various target datasets, showcasing its effectiveness in remote sensing image scene classification.

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfilment of the requirements for the M.Sc degree in Machine Learning

Advisors: Hisham Cholakkal, Fahad Khan

with 1 year embargo period

This document is currently not available here.

Share

COinS