Machine Learning

Why do over-parameterized networks generalize at all? What do they actually learn? Our machine learning research treats these as scientific questions — to be answered with careful experiments and, where possible, with theory.

Research topics

  • Representation learning
  • Optimization
  • Deep learning
  • Self-supervised learning
  • Generalization
  • Efficient learning
  • Model evaluation
Fig. 02Schematic — not experimental data
θ₀θ*loss surface · schematic
Fig. 02Schematic. A curved surface stands in for a learned representation space; the trajectory traces an optimizer descending toward a low-loss region.

Why it matters

Why it matters

Progress in machine learning has often run ahead of understanding. Practitioners know which architectures and training recipes work, but frequently not why — which makes it difficult to predict when they will stop working. A firmer scientific account of learning would make models more efficient to build, easier to diagnose, and more predictable when deployed.

Efficiency is part of the same question. If we understood what a model needs to learn, we could ask how little data, compute, or energy is required to learn it — and make capable systems accessible beyond the few organizations able to train at the largest scales.

Key research questions

Key research questions

  1. Q1

    Which properties of data and architecture determine whether a model learns a general rule or a brittle shortcut?

  2. Q2

    How do representations evolve during training, and can we identify the moments at which new capabilities emerge?

  3. Q3

    What makes self-supervised objectives produce representations that transfer across tasks?

  4. Q4

    Where does the efficiency of current training methods break down, and what would a more sample-efficient learner look like?

  5. Q5

    How should we measure generalization when test sets are no longer independent of training data?

Primary subfields

Primary subfields

Representation learning
How models build internal features, and what makes those features reusable.
Optimization
The geometry of loss landscapes and the implicit biases of training dynamics.
Generalization theory
Explaining when and why learned functions extend beyond the data they were fitted to.
Self-supervised learning
Learning structure from unlabelled data through prediction, reconstruction, and contrast.
Efficient learning
Reducing the data, compute, and memory required for a given level of capability.

Representative methodologies

Representative methodologies

  • M1

    Controlled synthetic tasks

    Small problems with known ground truth, used to isolate a single phenomenon.

  • M2

    Scaling studies

    Measuring how behavior changes with data, parameters, and compute.

  • M3

    Representation probing

    Analysing internal activations to test hypotheses about what a model encodes.

  • M4

    Ablation and replication

    Removing components and re-running experiments across seeds before drawing conclusions.

Relationships to other domains

Relationships to other domains

Working on questions in Machine Learning?

We welcome conversations with researchers and organizations working on related questions — whether to compare notes, discuss a collaboration, or share work in progress.