You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
I guarantee you that the dominant AI algorithm of the 2050s will not be a stack of differentiable layers trained with SGD.
At this point we are pretty close to training our language models on the entirety of text corpora that humanity has produced so far. Their language understanding capabilities are still close to zero.
But in 2050 imagine how much more text data we'll have and how many layers we can train 
. Jokes aside things will need to seriously change to get to the sci-fi levels everyone thinks are around the corner.
To think we can sum up the meaning of a word with 300 numbers we find by checking what words are next to it is kind of absurd.
Studies of the source of speedup across many areas has shown that roughly half of the improvement is due to hardware and half due to software. In some cases, algorithm advances have been much more important than hardware.pic.twitter.com/Zh3BOHsHra
I think the breakthrough might be more at the theoretical level. Most of NLP is driven by the distributional hypothesis at the moment. Similar low level research might yield fruit, instead of stacking more effort at the higher levels.
No we aren’t (re: closeness to maxing out the data). A few dozen gigs were used for GPT-2.
True that. GPT-2 was trained on 8M documents. Wayback Machine hosts 388B web pages.
This , I do think, because understanding human language serve no purpose for IA yet I mean, what could the loss function be for «understanding » human language ?
Mostov our langage modeling is based on cross entropy aka retreiving same distribution of words of the original corpus Even a seq2seq just find the most probable answer given distribution of the input sequence
@LordSomen True. I feel AI is so much more than current applications.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
