Gonçalo Oliveira, CAMGSD & Instituto Superior Técnico
Infinitely wide Neural Networks

I will explain how to think of infinitely wide neural networks at both initialization and during training. This means, its initial value and how it evolves along its training. At initialization, I will show that such neural networks are equivalent to a Gaussian process. During training, I will show that their evolution is equivalent to an autonomous linear flow in the space of functions. This is related to a phenomenon called (the lack of) feature learning and I intend to at least mention what that is.

Based on:

1. Luís Carvalho, João Lopes Costa, José Mourão, Gonçalo Oliveira. Wide neural networks: From non-gaussian random fields at initialization to the NTK geometry of training, arXiv:2304.03385.
2. L. Carvalho, J. L. Costa, J. Mourão, G. Oliveira. The positivity of the Neural Tangent Kernel, to appear in SIMODS (SIAM Journal on Mathematics of Data Science), arXiv:2404.12928.

Permanent link to this information: https://m4ai.math.tecnico.ulisboa.pt/lecture_series?sgid=84