Byzantine Tolerant Gradient Aggregation for Cross-Silo Federated Learning

Date of Award


Document Type


Degree Name

Master of Science in Computer Vision


Computer Vision

First Advisor

Dr. Karthik Nandakumar

Second Advisor

Dr. Mohammad Yaqub


Model poisoning is an extensively studied threat model in the federated learning paradigm. Though plenty of Byzantine robust aggregation methods show theoretical significance and empirical effectiveness, there exist quintessential issues with existing approaches when adapted for practical application. (i)~With heavy heterogeneity, median/outlier estimation methods become sub-optimal or fail to converge even in the absence of malicious actors i.e.~Byzantines. (ii)~Variation reduction approaches like clipping with bucketing strategies handle heterogeneity with provable convergence, but these inevitably lead to the mixing of harmful updates because of their formulation under harsher settings, leading to poisoning. Also, there are practical difficulties in determining the appropriate hyperparameters, such as assuming the number of Byzantines or setting clipping radius, which are functions of data heterogeneity, model parameter complexity, and optimization techniques. To overcome these limitations, we propose FedRISE aggregation, which uses sparsified gradients. The proposed method involves sign-voting to determine the individual gradients's optimal sign. We incorporate a reputation score for weighing each client's votes during sign selection. Our experiments show that existing robust aggregators fail under severe attacks, while FedRISE demonstrates better robustness under different attacks. In addition, we also introduce a data heterogeneity measurement method that is compliant with the differential privacy principles, which is helpful for a preliminary evaluation of heterogeneity among clients.


Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfilment of the requirements for the M.Sc degree in Computer Vision

Advisors: Karthik Nandakumar, Mohammad Yaqub

with 2 years embargo period

This document is currently not available here.