Deep Neural Nets: 33 years ago and 33 years from now
Source
- Source kind:
blog-rss-item - URL: http://karpathy.github.io/2022/03/14/lecun1989/
- Discovery source: https://karpathy.github.io/feed.xml
- License:
NOASSERTION - Distribution policy:
public-summary-local-archive-only - Public mirror status:
summary-only - Content hash:
47f509c6ab4f84912570a265e21999c4c367245949ca7b158f2b4a748bcd7747 - First seen: 2026-05-15
- Last changed: 2026-05-15
Classification
- Primary category: Personal heuristics / AI philosophy / learning advice
- Corpus source note: 2026-05-15-karpathy-public-corpus
- Project taxonomy: karpathy-project-taxonomy
- Idea map: karpathy-idea-map
- Topic hub: karpathy-public-work
Summary
The Yann LeCun et al. (1989) paper Backpropagation Applied to Handwritten Zip Code Recognition is I believe of some historical significance because it is, to my knowledge, the earliest real-world application of a neural net trained end-to-end with backpropagation. Except for the tiny dataset (7291 16x16 grayscale images of digits) and the tiny neural network used (only 1,000 neurons), this paper reads remarkably modern today, 33 years later - it lays out a dataset, describes the neural net architecture, loss function, optimization, and reports the experimental classification error rates over training and test sets. Itâs all very recognizable and type checks as a modern deep learning paper, except it is from 3…
What This Teaches
- How Karpathy frames technical judgment, learning, research, or AI systems in long-form prose.
- Useful as a high-signal idea source for research taste, project framing, and agent workflow design.
Why It Matters
Karpathy’s posts often crystallize reusable heuristics; this wiki should preserve the ideas without relying on chat memory.
Public Handling Notes
- EXTRACTED: This page records public metadata and a source-grounded summary.
- INFERRED: Full local preservation, when available, is for private/local use unless a license or explicit source policy makes public redistribution safe.
- Do not treat this page as permission to republish unlicensed source text or code wholesale.