Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study
Source
- Person key:
ysymyth - Source kind:
paper - Canonical URL: https://arxiv.org/abs/2403.08604
- License:
NOASSERTION - Public handling:
public-metadata-summary-hash-link-only - Semantic hash:
e4442c191ef9bcb65f262ace5887f23c35e7dfeeeadde6d0ea9b86c6385171f0 - First seen: 2026-05-16
- Last changed: 2026-05-16
- Identity guard: Do not confuse with yao-shunyu-alfred, the physics-to-AI researcher at alfredyao.github.io.
Classification
- Category: Software engineering agents and benchmarks
- Topic hub: shunyu-yao-public-corpora
- Project taxonomy: shunyu-yao-project-taxonomy
- Paper map: shunyu-yao-paper-map
Summary
Recent advancements in large language models (LLMs) have significantly enhanced their coding capabilities. However, existing benchmarks predominantly focused on simplified or isolated aspects of coding, such as single-file code generation or repository issue debugging, falling short of measuring the full spectrum of challenges raised by real-world programming activities. In this case study, we explore the performance of LLMs across the entire software development lifecycle with DevEval, encompassing stages includin…
What This Teaches
How coding agents are benchmarked, scaffolded, and constrained against real repositories and issues.
Related
Public Handling Notes
- EXTRACTED: Metadata and links are from public sources.
- INFERRED: Unclear-license full text, PDFs, and source code should not be mirrored into public wiki pages.
- AMBIGUOUS: Items discovered by keyword search are still separated by source identity and category heuristics.