Question: Is there a way to not break the orthogonality of weight matrices by backprop, given your n/w was initialized with othogonal Ws?
-
I see. How did you enforce orthogonality?
loss = sum of inner product of weights times scaling coefficient.
-
-
Isn't that just the frobenius norm? I was thinking in terms of min L(x) s.t. W_1^T W_1 = I, ... ,W_k^T W_k = I. But that's hard for me.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.