YOUR AD GOES HERE

How to compress text to 1 bit per character | Compression is Intelligence Part 3

Published 17, Nov 2026

3Blue1Brown


Description:
Arithmetic codes and how they elucidate this trilogy's title
Channel career fair: https://3b1b.com/talent
New store items! https://3b1b.com/store
Early views for supporters: https://3b1b.com/support

Other resources you may enjoy.

On Asymmetric numeral systems:
https://arxiv.org/abs/0902.0271
https://demonstrations.wolfram.com/DataCompressionUsingAsymmetricNumeralSystems/
https://arxiv.org/pdf/2201.01741v2
https://arxiv.org/pdf/2510.13867

On Arithmetic Codes:
https://dl.acm.org/doi/10.1145/214762.214771

Ilya Sutskever, An Observation on Generalization
https://www.youtube.com/watch?v=AKMuA_TVz3A&t=24s

A bit on prequential coding:
https://yidingjiang.github.io/blog/post/curriculum/

A few other good papers/posts on the relationship between compression, prediction, and ML:
https://deepmind.google/research/publications/39768/
https://www.hutter1.net/ai/uaibook.htm
https://link.springer.com/chapter/10.1007/3-540-62858-4_69
https://www.sciencedirect.com/science/article/abs/pii/0005109878900055

Timestamps:
0:00 - The absurdity of Shannon’s estimate
6:27 - From text to real numbers
16:57 - Is this Shannon’s limit?
20:26 - Reframing pretraining
22:41 - Arithmetic codes in action
27:29 - Reflections on “Compression is Intelligence”
36:30 - Three ways to support

Secret Enscreen Vlog: https://youtu.be/hoX3IiKXT-M

------------------

These animations are largely made using a custom Python library, manim. See the FAQ comments here:
https://3b1b.co/faq#manim

Music by Vincent Rubinetti.
https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u

------------------

3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly.

Mailing list: https://3blue1brown.substack.com
Twitter: https://twitter.com/3blue1brown
Bluesky: https://bsky.app/profile/3blue1brown.com
Instagram: https://www.instagram.com/3blue1brown
Reddit: https://www.reddit.com/r/3blue1brown
Facebook: https://www.facebook.com/3blue1brown
Patreon: https://patreon.com/3blue1brown
Website: https://www.3blue1brown.com

Releted More Videos

  • Sorry!!! Nothing to show

You May Also Like

YOUR AD GOES HERE

YOUR AD GOES HERE