Seminars

Europe/Lisbon
Room P3.10, Mathematics Building — Online

Gonçalo Oliveira, CAMGSD & Instituto Superior Técnico

Infinitely wide Neural Networks II

I will explain how to think of infinitely wide neural networks at both initialization and during training. This means, its initial value and how it evolves along its training. At initialization, I will show that such neural networks are equivalent to a Gaussian process. During training, I will show that their evolution is equivalent to an autonomous linear flow in the space of functions. This is related to a phenomenon called (the lack of) feature learning and I intend to at least mention what that is.

Based on:

  1. Luís Carvalho, João Lopes Costa, José Mourão, Gonçalo Oliveira. Wide neural networks: From non-gaussian random fields at initialization to the NTK geometry of training, arXiv:2304.03385.
  2. L. Carvalho, J. L. Costa, J. Mourão, G. Oliveira. The positivity of the Neural Tangent Kernel, to appear in SIMODS (SIAM Journal on Mathematics of Data Science), arXiv:2404.12928.