Training compute growth
Estimated training compute of frontier models by year (Epoch AI).
About this data
Estimated training compute for frontier models has grown several orders of magnitude, on the order of 4–5× per year per Epoch AI. The log axis makes the otherwise-exponential trend readable and underlines why power and chips have become the binding constraints.
Training compute growth
Estimated training compute per frontier release (log scale, FLOPs).
Data table
| year | series | source_ref | value_basis | flops_estimate |
|---|---|---|---|---|
| 2020 | training_compute | epoch-ai | GPT-3 ~3.1e23 FLOP (Epoch AI; corroborated by Stanford HAI AI Index) | 3.1e+23 |
| 2022 | training_compute | epoch-ai | PaLM-class frontier ~2.5e24 FLOP (Epoch AI) | 2.5e+24 |
| 2023 | training_compute | epoch-ai | GPT-4 ~2e25 FLOP (Epoch AI) | 2e+25 |
| 2024 | training_compute | epoch-ai | Gemini Ultra ~5e25 FLOP (Epoch AI) | 5e+25 |
| 2025 | training_compute | epoch-ai | Grok-4 ~5e26 FLOP, largest known run (Epoch AI) | 5e+26 |
Methodology & sources
Last updated: Aug 5, 2026Methodology
Source-backed values are seeded for four of the five charts: the release cadence by lab (2024 → 2026, from each lab’s release notes), the capability-vs-cost scatter (Artificial Analysis Intelligence Index vs output $/M tokens, June 2026 snapshot), training-compute growth by year (Epoch AI, corroborated by Stanford HAI), and a safety-gated capabilities matrix. Every numeric point carries a sources[].ref and a value_basis. Sources are each lab’s own model cards/release notes, Artificial Analysis, Epoch AI, and Stanford HAI — cross-checked against public release timelines. EDITORIAL ENCODING: the safety-gated matrix scores each model × domain as 3 = allowed / 2 = gated / 1 = blocked. This is an interpretation of each model’s published safety policy, not a measured benchmark. PLACEHOLDER: the per-lab benchmark-trajectory chart is left unseeded — a consistent historical per-quarter, per-lab benchmark series was not sourceable without mixing incompatible benchmarks. Re-verified 2026-06-15.
Sources
- Epoch AI — compute & training trends ↗ Open access
Comparisons are informative, not definitive. See each source for definitions and limits.