NN training goes through two distinct phases: (1) reducing classification error through drift (2) optimally compressing mutual information of hidden layers through diffusion https://arxiv.org/abs/1703.00810 pic.twitter.com/He7dNY3NSv
Better ways of thinking, feeling, and acting—around problems of meaning and meaninglessness; self and society; ethics, purpose, and value.
You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
Add this Tweet to your website by copying the code below. Learn more
Add this video to your website by copying the code below. Learn more
By embedding Twitter content in your website or app, you are agreeing to the Twitter Developer Agreement and Developer Policy.
| Country | Code | For customers of |
|---|---|---|
| United States | 40404 | (any) |
| Canada | 21212 | (any) |
| United Kingdom | 86444 | Vodafone, Orange, 3, O2 |
| Brazil | 40404 | Nextel, TIM |
| Haiti | 40404 | Digicel, Voila |
| Ireland | 51210 | Vodafone, O2 |
| India | 53000 | Bharti Airtel, Videocon, Reliance |
| Indonesia | 89887 | AXIS, 3, Telkomsel, Indosat, XL Axiata |
| Italy | 4880804 | Wind |
| 3424486444 | Vodafone | |
| » See SMS short codes for other countries | ||
This timeline is where you’ll spend most of your time, getting instant updates about what matters to you.
Hover over the profile pic and click the Following button to unfollow any account.
When you see a Tweet you love, tap the heart — it lets the person who wrote it know you shared the love.
The fastest way to share someone else’s Tweet with your followers is with a Retweet. Tap the icon to send it instantly.
Add your thoughts about any Tweet with a Reply. Find a topic you’re passionate about, and jump right in.
Get instant insight into what people are talking about now.
Follow more accounts to get instant updates about topics you care about.
See the latest conversations about any topic instantly.
Catch up instantly on the best stories happening as they unfold.
NN training goes through two distinct phases: (1) reducing classification error through drift (2) optimally compressing mutual information of hidden layers through diffusion https://arxiv.org/abs/1703.00810 pic.twitter.com/He7dNY3NSv
When I first saw this a few days ago I thought “peculiar —wonder if it’s true.” Looking at this graph, wondered what happened at 8-9k epochs (sudden dive).
Skimmed bits of paper, got increasingly suspicious that result is artifact of an artificial task. Went looking for the task… found this.pic.twitter.com/MmN7Lq6U3I
they mention later on that they also do the analysis for cases with no symmetries
the high signal -> high noise transition in training is also general afaict in the couple of DL projects I've tried, NNs do in general seem to keep improving after a clear signal is gone
Oh, well *that* is interesting. I would like to see it demonstrated on a real task. This paper has gotten a lot of hype, which doesn’t seem deserved.pic.twitter.com/eiulZpOhkK
I think the "hype" is cuz a well founded and motivated paradigm is being tried to explain neural black boxes. not that we gotten to the point of cool experimental verification.
Yes… I would reserve enthusiasm for the point at which there’s significant evidence. It’s easy to “apply” paradigms; hard to do so and get a meaningful result.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.