Representation Learning

Mar 3, 2022

How can we make sure that machines learn a good representation of the data? If labelling the data is a typical solution, it is often costly and time-consuming. Models such as neural networks typically require thousands to millions of examples to solve a task. Is it possible to learn good representations of the data without having to label each one of these examples?

Representation Learning provides interesting solutions through semi-supervised and self-supervised learning. With these techniques, it becomes increasingly realistic to solve complex tasks with few labelled examples. In some cases, we might want these representations to be understandable by human users. This induces a significant overlap with Explainable Artificial Intelligence.

Machine learning Deep learning Unsupervised Learning Semi-Supervised Learning Self-Supervised Learning Transfer Learning Few Shot Learning Generative AI

Jonathan Crabbé

PhD Researcher

My research focuses on explainable artificial intelligence, representation learning and robust machine learning.

Posts

2 Papers Accepted at ICML 2022

Label-Free Explainability and Data-SUITE have been accepted at ICML 2022. These are two new methods that machine learning interpretability and robustness. I provide a quick summary of the methods along with the relevant links.

Jonathan Crabbé

Jun 8, 2022 1 min read Conferences

2 Papers Accepted at ICML 2022

Publications

Robust multimodal models have outlier features and encode more concepts

We investigate what makes multimodal models that show good robustness with respect to natural distribution shifts (e.g., zero-shot CLIP) different from models with lower robustness using interpretability.

Jonathan Crabbé, Pau Rodríguez, Vaishaal Shankar, Luca Zappella, Arno Blaas

Robust multimodal models have outlier features and encode more concepts

TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization

We introduce TANGOS, a regularization method that orthogonalizes the gradient attribution of neurons to improve the generalization of deep neural networks on tabular data.

Alan Jeffares, Tennison Liu, Jonathan Crabbé, Fergus Imrie, Mihaela van der Schaar

Joint Training of Deep Ensembles Fails Due to Learner Collusion

Training deep ensembles via a shared objective results in degenerate behavior.

Alan Jeffares, Tennison Liu, Jonathan Crabbé, Mihaela van der Schaar

Joint Training of Deep Ensembles Fails Due to Learner Collusion

Concept Activation Regions: A Generalized Framework For Concept-Based Explanations

We extend existing feature and example importance methods to unsupervised learning.

Jonathan Crabbé, Mihaela van der Schaar

Concept Activation Regions: A Generalized Framework For Concept-Based Explanations

Label-Free Explainability for Unsupervised Models

We extend existing feature and example importance methods to unsupervised learning.

Jonathan Crabbé, Mihaela van der Schaar

Label-Free Explainability for Unsupervised Models