Conventional wisdom attributes the mysterious generalization abilities of overparameterized neural networks to gradient descent (and its variants). The recent volume hypothesis challenges this view: it posits that these generalization abilities persist even when gradient descent is replaced by Guess & Check (G&C), i.e., by drawing weight settings until one that fits the training data is found. The validity of the volume hypothesis for wide and deep neural networks remains an open question. In this talk, I will theoretically investigate this question for matrix factorization---a common testbed in neural network theory. I will first show that generalization under G&C deteriorates with increasing width, establishing what is, to my knowledge, the first case where G&C is provably inferior to gradient descent. Conversely, I will show that generalization under G&C improves with increasing depth, revealing a stark contrast between wide and deep networks, which is validated empirically. These findings suggest that even in simple settings, there may not be a simple answer to the question of whether neural networks need gradient descent to generalize well.

I will discuss some connections between random matrix theory and machine learning, focusing on the spectrum of the hessian of the loss surface and touching on issues relating to neural collapse.

TBA

Deep neural networks have become the cornerstone of modern machine learning, yet their multi-layer structure, nonlinearities, and intricate optimization processes pose considerable theoretical challenges. In this talk, I will review recent advances in random matrix analysis that shed new light on these complex ML models. Starting with the foundational case of linear regression, I will demonstrate how the proposed analysis extends naturally to shallow nonlinear and ultimately deep nonlinear network models. I will also discuss practical implications (e.g., compressing and/or designing "equivalent" NN models) that arise from these theoretical insights. The talk is based on a recent review paper https://arxiv.org/abs/2506.13139 joint with Michael W. Mahoney.

TBA