The Hopfield model stands as a paradigm at the intersection of statistical physics, theoretical neuroscience, and machine learning. Originally introduced as a biologically inspired model of associative memory, it has since evolved into a foundational framework for understanding a wide range of complex systems.

On the one hand, its roots in neuroscience enable a fruitful cross-fertilization: biologically grounded mechanisms continue to inspire algorithmic refinements and performance improvements in modern associative memory models. On the other hand, its formal connection with Boltzmann machines provides a bridge to contemporary machine learning techniques, including strategies such as dropout, pre-training, and the optimization of activation functions.

From the perspective of statistical mechanics, the Hopfield model remains a cornerstone for the analytical study of high-dimensional systems with disorder and frustration. This viewpoint naturally extends to the investigation of structured datasets, where the model offers a tractable yet expressive starting point for developing analytical insights.

In this talk, after a gentle introduction to the model, we will highlight some of these current research directions, while keeping the presentation accessible to a non-technical audience.

Existing machine learning frameworks operate over the field of real numbers ($\mathbb{R}$) and learn representations in real (Euclidean or Hilbert) vector spaces (e.g., $\mathbb{R}^d$). Their underlying geometric properties align well with intuitive concepts such as linear separability, minimum enclosing balls, and subspace projection; and basic calculus provides a toolbox for learning through gradient-based optimization.

But is this the only possible choice? In this seminar, we study the suitability of a radically different field as an alternative to $\mathbb{R}$ — the ultrametric and non-archimedean space of $p$-adic numbers, $\mathbb{Q}_p$. The hierarchical structure of the $p$-adics and their interpretation as infinite strings make them an appealing tool for code theory and hierarchical representation learning. Our exploratory theoretical work establishes the building blocks for classification, regression, and representation learning with the $p$-adics, providing learning models and algorithms. We illustrate how simple Quillian semantic networks can be represented as a compact $p$-adic linear network, a construction which is not possible with the field of reals. We finish by discussing open problems and opportunities for future research enabled by this new framework.

Based on:
André F. T. Martins, Learning with the $p$-adics

Diffusion probabilistic models have become the state-of-the-art tool in generative methods, used to generate high-resolution samples from very high-dimension distributions (e.g. images). Although very effective, they suffer some drawbacks:

1. as opposed to variational encoders, the dimension of the problem remains high during the generation process and
2. they can be prone to memorization of the training dataset.

In this talk, we first provide an introduction to generative modeling, with a focus on diffusion models from the point of view of stochastic PDEs. Then, we introduce a kernel-smoothed empirical score and study the bias-variance of this estimator. We find improved bounds on the KL-divergence between a true measure and an approximate measure generated by using the smoothed empirical score. This score estimator leads to less memorization and better generalization. We demonstrate these findings on synthetic and real datasets, combining diffusion models with variational encoders to reduce the dimensionality of the problem.