build-nanogpt

Source

• Source kind: github-repository
• URL: https://github.com/karpathy/build-nanogpt
• Discovery source: https://github.com/karpathy/build-nanogpt
• License: NOASSERTION
• Distribution policy: public-summary-local-archive-only
• Public mirror status: summary-only
• Content hash: e6d96291388c3994f988b38656786ecbcb357910005fa1606bba2ff9fe9a2ff0
• First seen: 2026-05-15
• Last changed: 2026-05-15

Classification

• Primary category: LLM training and inference systems
• Corpus source note: 2026-05-15-karpathy-public-corpus
• Project taxonomy: karpathy-project-taxonomy
• Idea map: karpathy-idea-map
• Topic hub: karpathy-public-work

Summary

build nanoGPT This repo holds the from-scratch reproduction of $1.Thegitcommitswerespecificallykeptstepbystepandcleansothatonecaneasilywalkthroughthegitcommithistorytoseeitbuiltslowly.Additionally,thereisanaccompanying$1 where you can see me introduce each commit and explain the pieces along the way. We basically start from an empty file and work our way to a reproduction of the $1(124M)model.Ifyouhavemorepatienceormoney,thecodecanalsoreproducethe$1 models. While the GPT-2 (124M) model probably trained for quite some time back in the day (2019, ~5 years ago), today…

What This Teaches

• How modern LLM training or inference can be reduced to compact, inspectable systems.
• Useful as a reference for building mental models of GPT-style models without hiding behind framework scale.

Why It Matters

This is high-priority for Vipin because it connects directly to LLM systems, evaluation, and research implementation judgment.

Repository Snapshot

• Full name: karpathy/build-nanogpt
• Default branch: master
• HEAD: 6104ab1b53920f6e2159749676073ff7d815c1fa
• Stars at crawl: 5009
• Forks at crawl: 802
• File count: 6
• README path: README.md
• License path: “
• Created: 2024-06-09T05:16:13Z
• Updated: 2026-05-15T19:16:47Z
• Pushed: 2024-08-13T12:28:52Z

Top-Level Structure

• [root]: 6

File Extension Profile

• .py: 3
• .ipynb: 1
• .md: 1
• .txt: 1

Tags / Release-Like Markers

• No git tags found in the shallow local clone.

Sample File Tree

• fineweb.py
• hellaswag.py
• input.txt
• play.ipynb
• README.md
• train_gpt2.py

Public Handling Notes

• EXTRACTED: This page records public metadata and a source-grounded summary.
• INFERRED: Full local preservation, when available, is for private/local use unless a license or explicit source policy makes public redistribution safe.
• Do not treat this page as permission to republish unlicensed source text or code wholesale.