Floored by how excellent this conversation between and is. Definitely worth the time to watch.
These two have beautiful minds and, even more importantly, really great hearts.
youtu.be/13CZPWmke6A
13
54
502
Conversation
On deep learning:
“The geometric mean of biology and physics... that’s deep learning”
1
22
On transformers:
Transformers are successful because they are a simultaneous combination of multiple ideas: attention, ability to run quickly on a GPU, non-recurrence (and therefore less deep so easier to optimize)
1
15
On how to build AGI:
Best guess is it will be deep learning plus a few other smaller ideas (self-play among them)
1
1
12
On self play:
“Self-play has a property that it can surprise us in truly novel ways.” (e.g., creative solutions to problems that weren’t anticipated — and are actually useful!)
4
1
33
