Some Results on the Expressivity of Deep Neural Networks
Deep Neural Networks (NNs), have attracted a lot of attention since the year 2000, thanks to their impressive state-of-the-art performance in the machine learning field and its tasks. In this thesis, we study the expressivity of deep neural networks, especially the number of linear regions for PPNNs and Piecewise Polynomial Convolutional Neural Networks (PPCNNs). Our results suggest that, deep PPCNNs have way more of an expressivity than shallower PPCNNs having equal number of parameters; and deep PPCNNs have more expressivity than deep PPNNs under some wild architectural assumptions.
M.J.R. Alzaabi, "Some Results on the Expressivity of Deep Neural Networks", M.S. Thesis, Machine Learning, MBZUAI, Abu Dhabi, UAE, 2022.