Wide neural networks can be seen as random functions whose behavior simplifies in several different large-width regimes. At the scale of typical fluctuations, central-limit phenomena lead to Gaussian-process limits with recursively defined covariance kernels, while the parameter-space tangent geometry converges to the neural tangent kernel and yields a linearized description of training. These Gaussian and kernel limits provide a remarkably tractable theory of random initialization and learning, but they capture only part of the finite-width behavior.
The course develops a probabilistic view of random wide neural networks across these different scales. We will study Gaussian and quantitative finite-width approximations, neural tangent kernel limits, large deviations of outputs and empirical layer kernels, and regimes in which feature learning survives at large width. Large-deviation theory provides a complementary description of rare, non-Gaussian behavior: atypical outputs can be realized through atypical internal kernels, leading to variational representations of the output rate function. Under suitable Bayesian scalings, the same viewpoint yields posterior variational problems over both predictors and kernels, providing one probabilistic mechanism for data-dependent feature learning. We will compare this with dynamical feature-learning limits such as mean-field and maximal-update regimes, emphasizing throughout how different scalings reveal different aspects of neural networks.

TBA

TBA

TBA

TBA