We compared different LLMs on IMO 2026 [R]

There are a few reasons why problems from International Mathematical Olympiad function as a good benchmark for LLMs: - The problems are new, not included in the training data of any model - Hard math problems are quite a good proxy for general intelligence capability - These are complex multi-step tasks that can benefit from orchestration / harness engineering Results: Frontier models (sol and fable) were able to get perfect / nearly perfect score regardless of harness. For both sonnet and opus, the webapp performance was quite poor, improved by provider harness (claude code) and even further