Claude Shannon English Text Compression
2 videos · score: 6,901 · first seen Jun 18, 2026
🟢 TRENDING UPYouTube creators are discussing Claude Shannon's 1950 theory that English text can be compressed to about one bit per character with sufficient context — 3Blue1Brown highlights how modern compression techniques now approach this limit through engineered models rather than human intuition, while Fireship traces a broader historical chain linking Shannon’s work to modern AI and Google’s founding paper, emphasizing its role in the rise of billion-dollar startups and large language models.
I read every major CS paper of the last 100 years...
A retrospective video traces a 100-year chain from Turing's 1936 computability paper to Google's 1998 PageRank paper, arguing that specific breakthroughs—including 1986 backpropagation—directly enabled modern AI like LLMs and billion-dollar startups.
Measuring the entropy of English
Seventy-five years after Claude Shannon estimated that English text could be compressed to about one bit per character with sufficient context, modern compression techniques are now achieving this theoretical limit through engineering intelligence rather than human intuition.