How many different problems can a neural network solve? What makes two machine learning problems different? In this talk, we'll show how Topological Data Analysis (TDA) can be used to partition classification problems into equivalence classes, and how the complexity of decision boundaries can be quantified using persistent homology. Then we will look at a network's learning process from a manifold disentanglement perspective. We'll demonstrate why analyzing decision boundaries from a topological standpoint provides clearer insights than previous approaches. We use the topology of the decision boundaries realized by a neural network as a measure of a neural network's expressive power. We show how such a measure of expressive power depends on the properties of the neural networks' architectures, like depth, width and other related quantities.

References

• Ballester et al, Topological Data Analysis for Neural Network Analysis: A Comprehensive Survey
• Papamrakou et al, Position: Topological Deep Learning is the New Frontier for Relational Learning
• Petri and Leitão, On The Topological Expressive Power of Neural Networks

Zoom: https://tecnico-pt.zoom.us/j/93935874388?pwd=QHxbpTCtH00rY4OUsRaay48CgaglgB.1

How many different problems can a neural network solve? What makes two machine learning problems different? In this talk, we'll show how Topological Data Analysis (TDA) can be used to partition classification problems into equivalence classes, and how the complexity of decision boundaries can be quantified using persistent homology. Then we will look at a network's learning process from a manifold disentanglement perspective. We'll demonstrate why analyzing decision boundaries from a topological standpoint provides clearer insights than previous approaches. We use the topology of the decision boundaries realized by a neural network as a measure of a neural network's expressive power. We show how such a measure of expressive power depends on the properties of the neural networks' architectures, like depth, width and other related quantities.

References

• Ballester et al, Topological Data Analysis for Neural Network Analysis: A Comprehensive Survey
• Papamrakou et al, Position: Topological Deep Learning is the New Frontier for Relational Learning
• Petri and Leitão, On The Topological Expressive Power of Neural Networks

Zoom: https://tecnico-pt.zoom.us/j/93935874388?pwd=QHxbpTCtH00rY4OUsRaay48CgaglgB.1

Traditional neural networks prioritize generalization, but this flexibility often leads to geometrically inconsistent transformations of input data. To account for variations in object pose — such as rotations or translations — models are typically trained on large, augmented datasets. This increases computational cost and complicates learning.

We propose an alternative: neural networks that are inherently invariant or equivariant to geometric transformations by design. Such models would produce consistent outputs regardless of an object’s pose, eliminating the need for data augmentation. This approach can potentially extend to a broad range of transformations beyond just rotation and translation.

To realize this, we use geometric algebra, where operations like the geometric product are naturally equivariant under pseudo-orthogonal transformations, represented by the group SO(4,1). By building neural networks on top of this algebra, we can ensure transformation-aware computation.

Additionally, we address permutation invariance in point clouds. Instead of treating them as unordered sets of vectors, we represent them functionally — as sums of Dirac delta functions — analogous to sampled signals. This avoids point ordering issues entirely and offers a more structured geometric representation.

This leads us to functional neural networks, where the input is a function rather than a vector list, and layers are continuous operators rather than discrete ones like ReLU or linear layers. Constructed within geometric algebra, these networks naturally maintain the desired invariant and equivariant properties.

Traditional neural networks prioritize generalization, but this flexibility often leads to geometrically inconsistent transformations of input data. To account for variations in object pose — such as rotations or translations — models are typically trained on large, augmented datasets. This increases computational cost and complicates learning.

We propose an alternative: neural networks that are inherently invariant or equivariant to geometric transformations by design. Such models would produce consistent outputs regardless of an object’s pose, eliminating the need for data augmentation. This approach can potentially extend to a broad range of transformations beyond just rotation and translation.

To realize this, we use geometric algebra, where operations like the geometric product are naturally equivariant under pseudo-orthogonal transformations, represented by the group SO(4,1). By building neural networks on top of this algebra, we can ensure transformation-aware computation.

Additionally, we address permutation invariance in point clouds. Instead of treating them as unordered sets of vectors, we represent them functionally — as sums of Dirac delta functions — analogous to sampled signals. This avoids point ordering issues entirely and offers a more structured geometric representation.

This leads us to functional neural networks, where the input is a function rather than a vector list, and layers are continuous operators rather than discrete ones like ReLU or linear layers. Constructed within geometric algebra, these networks naturally maintain the desired invariant and equivariant properties.

This lecture first provides an introduction to classical variational inference (VI), a key technique for approximating complex posterior distributions in Bayesian methods, typically by minimizing the Kullback-Leibler (KL) divergence. We'll discuss its principles and common uses.

Building on this, the lecture introduces Fenchel-Young variational inference (FYVI), a novel generalization that enhances flexibility.FYVI replaces the KL divergence with broader Fenchel-Young (FY) regularizers, with a special focus on those derived from Tsallisentropies. This approach enables learning posterior distributions with significantly smaller, or sparser, support than the prior, offering advantages in model interpretability and performance.

S. Sklavidis, S. Agrawal, A. Farinhas, A. Martins and M. Figueiredo, Fenchel-Young Variational Learning,
https://arxiv.org/pdf/2502.10295