1/ Random thought: Deep Learning model development has seemed to me to be a lot like cooking. Typical example: Images looking a little strange here? Train on this other loss to tone it down a bit. How much? Until it's done!
Show this thread
I personally lean heavily on experiments because to me the answers often emerge that way. I really try to be disciplined about being skeptical about all assumptions- especially about stuff I remember "ruling out" or "seeing", etc. But also what others have "ruled" out.
