InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback
Source
- Person key:
ysymyth - Source kind:
paper - Canonical URL: https://arxiv.org/abs/2306.14898
- License:
NOASSERTION - Public handling:
public-metadata-summary-hash-link-only - Semantic hash:
ae4ad1717a363ab61dc457dd60dbf85aeeb985cd196fe7c642080268b4128a89 - First seen: 2026-05-16
- Last changed: 2026-05-16
- Identity guard: Do not confuse with yao-shunyu-alfred, the physics-to-AI researcher at alfredyao.github.io.
Classification
- Category: Software engineering agents and benchmarks
- Topic hub: shunyu-yao-public-corpora
- Project taxonomy: shunyu-yao-project-taxonomy
- Paper map: shunyu-yao-paper-map
Summary
Humans write code in a fundamentally interactive manner and rely on constant execution feedback to correct errors, resolve ambiguities, and decompose tasks. While LLMs have recently exhibited promising coding capabilities, current coding benchmarks mostly consider a static instruction-to-code sequence transduction process, which has the potential for error propagation and a disconnect between the generated code and its final execution environment. To address this gap, we introduce InterCode, a lightweight, flexible…
What This Teaches
How coding agents are benchmarked, scaffolded, and constrained against real repositories and issues.
Related
Public Handling Notes
- EXTRACTED: Metadata and links are from public sources.
- INFERRED: Unclear-license full text, PDFs, and source code should not be mirrored into public wiki pages.
- AMBIGUOUS: Items discovered by keyword search are still separated by source identity and category heuristics.