In The Art of Doing Science and Engineering, Hamming gives this amazing sigmoidal formulation for the growth rate of computing power: e^(22(1-e^(-t/20))), with t=0 in 1943. That predicts 2.2 GHz for 2019, with is rather remarkably close to where we are.
-
Show this thread
-
Replying to @patrickc
And also remarkably close to the irritating fact that clock speeds seem to have just barely moved over the past 10 years.
7 replies 0 retweets 33 likes -
Replying to @michael_nielsen @patrickc
A famous Silicon designer once told me "all speed comes from parallelism" & power efficiency as well. Clock speed by itself can be a VERY deceptive measure. Power usage increases linearly with parallelism, but exponentially with clockrate. Speed increases linearly with both.
1 reply 2 retweets 10 likes -
Replying to @DavidSHolz @patrickc
That's been true for 10+ years, since clock speeds have largely saturated. It's quite certain that much of the speed increase between 1980 and 2010 came from clock speed increase (itself enabled by other things, e.g., smaller feature sizes, as I understand it).
1 reply 0 retweets 2 likes -
Replying to @michael_nielsen @patrickc
How much could be enabled by faster clock-rates vs parallelism? Photo/video/audio, ML/ CV, AR/VR seem mostly about parallelism. Increases in parallel compute for AI has been mind-boggling. Beyond what I'd dream for clockrates. Faster clocks could make faster Javascript... Hmmm...
3 replies 1 retweet 2 likes -
Replying to @DavidSHolz @patrickc
Well, personally I've believed since the 80s that parallelism will win in the end. But I've been repeatedly too optimistic (i.e., flat wrong) about how rapidly that will happen.
1 reply 0 retweets 0 likes -
My guess is that eventually fairly parallelizable matrix-based ML methods mostly subsume what we currently think of as general-purpose computation. But right now you're not going to replace your single-threaded C or Javascript code with anything that looks like a neural net...
2 replies 1 retweet 1 like
It's a fun problem to think about - writing a web browser which is a neural net. (A friend of mine once taught his spam filter to play chess.)
-
-
Replying to @michael_nielsen
It might be a easier problem than writing a new web rendering engine from scratch! Very fun to think about
, lots of training data, probably faster and lower power too with the right Silicon
0 replies 0 retweets 1 likeThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.