Nuclei Segmentation and Classification

Document Type



Nuclei segmentation and classification is a very crucial step in computational pathology. However, this topic tends to face several challenges, including cluttered nuclei, identifying the subtle differences between nucleus classes, the imbalance between the different classes and between the foreground and background, the massive variety in the nuclei morphology, and finally staining. This work addresses some of these difficulties especially dealing with the variety of morphology of cells, imbalance, and extracting more fine-grained information between the different nucleus types. Thus, it is divided into two contribution chapters. The first contribution focuses on extracting more meaningful and insightful information through various techniques such as the usage of ConvNeXt as an encoder for HoVerNet architecture, utilizing the different color spaces for nuclei fine feature extraction, and finally employing affine augmentation, and unified focal loss. The second involves localizing each nucleus using a detection network. Then, all the detected nuclei and their corresponding bounding boxes are used to create a single cell image set that contains one nucleus per image with its respective mask. This single cell image set is used to train a multi-class segmentation model to segment each nucleus independently. Finally, the segmented nuclei from the same original image undergo a consistency check and correction block before being put together to obtain the final mask for the full image. The division of multi-class segmentation task into multiple simpler problems makes it more suitable to tackle each obstacle independently. To evaluate the performance of the first contribution, all the experiments were done on HoVer-Net, in which 2.8% and 3.2% increases were observed for both mPQ, and R^2, respectively. Furthermore, the second contribution resulted in having a more transparent architecture which eventually led to improving the performance of most classes compared to its initial U-Net baseline, while outperforming SOTA HoVer-Net on one of the highly imbalanced classes by 5.2% while performing competitively well on the other classes.

First Page


Last Page


Publication Date



Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfillment of the requirements for the M.Sc degree in Computer Vision

Advisors: Dr. Kun Zhang, Dr. Hang Dai

with 2 years embargo period

This document is currently not available here.