Can Language Models Solve Olympiad Programming?
Source
- Person key:
ysymyth - Source kind:
paper - Canonical URL: https://arxiv.org/abs/2404.10952
- License:
NOASSERTION - Public handling:
public-metadata-summary-hash-link-only - Semantic hash:
67724628408eb6129455346e5b425458902bc641637e59535521b2813c3d5e69 - First seen: 2026-05-16
- Last changed: 2026-05-16
- Identity guard: Do not confuse with yao-shunyu-alfred, the physics-to-AI researcher at alfredyao.github.io.
Classification
- Category: Evaluation / benchmark design
- Topic hub: shunyu-yao-public-corpora
- Project taxonomy: shunyu-yao-project-taxonomy
- Paper map: shunyu-yao-paper-map
Summary
Computing olympiads contain some of the most challenging problems for humans, requiring complex algorithmic reasoning, puzzle solving, in addition to generating efficient code. However, it has been understudied as a domain to evaluate language models (LMs). In this paper, we introduce the USACO benchmark with 307 problems from the USA Computing Olympiad, along with high-quality unit tests, reference code, and official analyses for each problem. These resources enable us to construct and test a range of LM inference…
What This Teaches
How to turn agent behavior into measurable tasks, success criteria, and repeatable benchmark environments.
Related
Public Handling Notes
- EXTRACTED: Metadata and links are from public sources.
- INFERRED: Unclear-license full text, PDFs, and source code should not be mirrored into public wiki pages.
- AMBIGUOUS: Items discovered by keyword search are still separated by source identity and category heuristics.