Document Type
Article
Publication Title
Bioengineering
Abstract
Medical image segmentation is a vital healthcare endeavor requiring precise and efficient models for appropriate diagnosis and treatment. Vision transformer (ViT)-based segmentation models have shown great performance in accomplishing this task. However, to build a powerful backbone, the self-attention block of ViT requires large-scale pre-training data. The present method of modifying pre-trained models entails updating all or some of the backbone parameters. This paper proposes a novel fine-tuning strategy for adapting a pretrained transformer-based segmentation model on data from a new medical center. This method introduces a small number of learnable parameters, termed prompts, into the input space (less than 1% of model parameters) while keeping the rest of the model parameters frozen. Extensive studies employing data from new unseen medical centers show that the prompt-based fine-tuning of medical segmentation models provides excellent performance regarding the new-center data with a negligible drop regarding the old centers. Additionally, our strategy delivers great accuracy with minimum re-training on new-center data, significantly decreasing the computational and time costs of fine-tuning pre-trained models. Our source code will be made publicly available.
DOI
10.3390/bioengineering10070879
Publication Date
7-24-2023
Keywords
limited data, medical image segmentation, prompt-based tuning, transfer learning, vision transformer
Recommended Citation
N. Saeed et al., "Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer," Bioengineering, vol. 10, no. 7, Jul 2023.
The definitive version is available at https://doi.org/10.3390/bioengineering10070879
Comments
Archived thanks to MDPI
License: CC BY 4.0
Uploaded: May 14, 2024