How would this perform compared to training a new network with the same architecture?
In new work with @KordingLab, we reconstruct the architecture, weights, and biases of unknown deep ReLU networks by observing their output. ReLU nets are piecewise linear functions - a lot of info is in the boundaries between linear regions.
Full paper: https://arxiv.org/abs/1910.00744 pic.twitter.com/QlhucOcb58
-
-
-
Well, this paper discovers the architecture instead of requiring knowledge of it. But also, my hunch is that the kind of self-distillation technique you suggest would not converge to the real weights in a multi layer perceptron. It is a very nonlinear problem.
Kraj razgovora
Novi razgovor -
-
-
This is awesome work!
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
@guillefix Check this out!Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.