AI Safety

Capability without understanding is a liability. Our safety research develops the tools to look inside models, evaluate them rigorously, and characterize how they behave when conditions change — so that progress can be matched by justified confidence.

Research topics

  • Interpretability
  • Alignment
  • Robustness
  • Model evaluation
  • Distribution shift
  • Uncertainty
  • Responsible deployment
Fig. 06Schematic — not experimental data
observedextrapolationextrapolation±2σ
Fig. 06Schematic. A model's prediction with an uncertainty band that widens away from observed data — the behavior a well-calibrated system should exhibit.

Why it matters

Why it matters

A model that performs well on a benchmark may still fail in deployment, misrepresent its own confidence, or pursue an objective that diverges subtly from what was intended. These problems become harder to detect as systems become more capable, because their outputs become harder for people to check directly.

We regard safety and capability as connected rather than opposed. The methods that let us understand a model — interpretability, careful evaluation, calibrated uncertainty — are the same methods that let us improve it with confidence. Safety research is therefore integral to our work across every other domain.

Key research questions

Key research questions

  1. Q1

    Can we identify, from a model's internals, the features and circuits responsible for a specific behavior?

  2. Q2

    How can evaluations remain informative as models approach or exceed the abilities of their evaluators?

  3. Q3

    When should a model abstain, and how can we make its confidence a trustworthy signal?

  4. Q4

    How do models behave under distribution shift, and can we detect shift before it causes failure?

  5. Q5

    What training methods produce behavior that remains aligned with intent in situations not covered by training?

Primary subfields

Primary subfields

Interpretability
Explaining model behavior in terms of internal representations and computation.
Evaluation
Designing measurements that remain valid, hard to game, and predictive of real behavior.
Robustness
Behavior under adversarial inputs and distribution shift.
Uncertainty quantification
Calibration, abstention, and honest reporting of confidence.
Alignment
Methods for specifying intent and checking that systems pursue it.

Representative methodologies

Representative methodologies

  • M1

    Mechanistic interpretability

    Reverse-engineering circuits and features in trained networks.

  • M2

    Red-teaming and adversarial testing

    Searching systematically for inputs that cause failure.

  • M3

    Calibration analysis

    Comparing stated confidence with observed accuracy across conditions.

  • M4

    Held-out and dynamic evaluation

    Test sets constructed to resist contamination and overfitting.

Relationships to other domains

Relationships to other domains

Working on questions in AI Safety?

We welcome conversations with researchers and organizations working on related questions — whether to compare notes, discuss a collaboration, or share work in progress.