This is fascinating: Rich Sutton on the "bitter lesson" of AI research: incompleteideas.net/IncIdeas/Bitte
28
116
572
Conversation
I think it's actually wrong in some important ways - Deep Blue was far from brute force alone. It involved > 8,000 features, many of them hand-engineered on the basis of deep chess expertise, exactly the kind of thing he dismisses.
Still, it's a very striking set of assertions
8
1
50
Nice in combination with : twitter.com/karpathy/statu
Also: Moore's Law means gradient descent gets better every year. So even if it's not a better programmer than you this year, it'll be much better next year.
Quote Tweet
Gradient descent can write code better than you. I'm sorry.
2
22
I'm sympathetic to Sutton's argument, but to try to make the other case: computers have done a _lot_ in the more symbolic mode. No-one is using neural nets to learn an OS, or to learn a database, or anything like that. Sutton's analysis is, in this sense, cherrypicked
4
34
(And in 3, 2, 1... I'm sure someone will give me an example of a neural net executing SQL queries or something like that :-) )
4
21
Replying to
I don't know of anyone training neural nets to execute SQL queries, but they now do better at planning them (i.e., turning the declarative query into an imperative plan) than PostgreSQL and even some commercial-grade optimizers. files.zotero.net/eyJleHBpcmVzIj
1
Traditional query optimizers feel like the most successful commercialization of 80s-style symbolic AI (although I am not an expert on this history here).
The performance improvement is modest but remarkable. "ComSys" is usually a system from a vendor like Oracle that doesn't want its name used in these benchmarking comparisons. These kinds of optimizers are built by, like, hundreds of experts over decades.
