A Recipe for Training Neural Networks
Source
- Source kind:
blog-rss-item - URL: http://karpathy.github.io/2019/04/25/recipe/
- Discovery source: https://karpathy.github.io/feed.xml
- License:
NOASSERTION - Distribution policy:
public-summary-local-archive-only - Public mirror status:
summary-only - Content hash:
99e97a0f8a3887488a09561c656874225f479162ff34914df54fc2a833425cd8 - First seen: 2026-05-15
- Last changed: 2026-05-15
Classification
- Primary category: Personal heuristics / AI philosophy / learning advice
- Corpus source note: 2026-05-15-karpathy-public-corpus
- Project taxonomy: karpathy-project-taxonomy
- Idea map: karpathy-idea-map
- Topic hub: karpathy-public-work
Summary
Some few weeks ago I posted a tweet on âthe most common neural net mistakesâ, listing a few common gotchas related to training neural nets. The tweet got quite a bit more engagement than I anticipated (including a webinar :)). Clearly, a lot of people have personally encountered the large gap between âhere is how a convolutional layer worksâ and âour convnet achieves state of the art resultsâ. So I thought it could be fun to brush off my dusty blog to expand my tweet to the long form that this topic deserves. However, instead of going into an enumeration of more common errors or fleshing them out, I wanted to dig a bit deeper and talk about how one can avoid making these errors altogether (or fix th…
What This Teaches
- How Karpathy frames technical judgment, learning, research, or AI systems in long-form prose.
- Useful as a high-signal idea source for research taste, project framing, and agent workflow design.
Why It Matters
Karpathy’s posts often crystallize reusable heuristics; this wiki should preserve the ideas without relying on chat memory.
Public Handling Notes
- EXTRACTED: This page records public metadata and a source-grounded summary.
- INFERRED: Full local preservation, when available, is for private/local use unless a license or explicit source policy makes public redistribution safe.
- Do not treat this page as permission to republish unlicensed source text or code wholesale.