Machine Learning
Why do over-parameterized networks generalize at all? What do they actually learn? Our machine learning research treats these as scientific questions — to be answered with careful experiments and, where possible, with theory.
- Representation learning
- Optimization
- Deep learning
- Self-supervised learning
- Generalization
- Efficient learning
- Model evaluation
Why it matters
Progress in machine learning has often run ahead of understanding. Practitioners know which architectures and training recipes work, but frequently not why — which makes it difficult to predict when they will stop working. A firmer scientific account of learning would make models more efficient to build, easier to diagnose, and more predictable when deployed.
Efficiency is part of the same question. If we understood what a model needs to learn, we could ask how little data, compute, or energy is required to learn it — and make capable systems accessible beyond the few organizations able to train at the largest scales.
Key research questions
Which properties of data and architecture determine whether a model learns a general rule or a brittle shortcut?
How do representations evolve during training, and can we identify the moments at which new capabilities emerge?
What makes self-supervised objectives produce representations that transfer across tasks?
Where does the efficiency of current training methods break down, and what would a more sample-efficient learner look like?
How should we measure generalization when test sets are no longer independent of training data?
Primary subfields
- Representation learning
- How models build internal features, and what makes those features reusable.
- Optimization
- The geometry of loss landscapes and the implicit biases of training dynamics.
- Generalization theory
- Explaining when and why learned functions extend beyond the data they were fitted to.
- Self-supervised learning
- Learning structure from unlabelled data through prediction, reconstruction, and contrast.
- Efficient learning
- Reducing the data, compute, and memory required for a given level of capability.
Representative methodologies
Controlled synthetic tasks
Small problems with known ground truth, used to isolate a single phenomenon.
Scaling studies
Measuring how behavior changes with data, parameters, and compute.
Representation probing
Analysing internal activations to test hypotheses about what a model encodes.
Ablation and replication
Removing components and re-running experiments across seeds before drawing conclusions.
Relationships to other domains
- AI Safety
Understanding representations is the foundation of interpretability.
- Generative AI
Foundation models are the largest-scale test of learning theory.
- Scientific Discovery
Scientific data stresses generalization in ways web-scale data does not.
Working on questions in Machine Learning?
We welcome conversations with researchers and organizations working on related questions — whether to compare notes, discuss a collaboration, or share work in progress.