François CholletVerified account

About SARM. At this point I am 100% convinced that the VGG16 experiment is not for real. Most likely a big experimental mistake, not fraud.

The paper is difficult to follow, so at first I wasn't sure about algo details. After some discussions, in particular with Kevin Murphy,

it seems that the VGG16 experiment merely consists in using convolutional kernels obtained by computing the PCA/LDA of the input filter maps

I have tried this exact same setup last year, explored every possible variant of that algorithm. I know for a fact that it doesn't work.

Better: I know why it doesn't work. Which isn't addressed in the paper.

Conditional on this interpretation of the algorithm used, there is a 0% chance that performance would be anywhere near backprop for VGG.

I cannot speak for the first two experiments. As for the VGG16 experiment, I am confident enough about this to state it publicly.

That said. If this paper manages to reignite interest in sparse coding, then that's a positive result. Important, under-appreciated topic.