A model compresses a state space by capturing a set of invariances that predict the variance in the states. Its free parameters define the latent space of the model and should ideally fully correspond to the variability, the not-invariant (= unexplained) remainder of the state.
Noise is incompressible information, i.e. information that you cannot relate to other information, such as nerves randomly firing. Uncertainty is estimated difference between model and ground truth. This includes uncertainty about the model quality (i.e. the model of your model).
-
-
So noise is where the function cannot resolve particular patterns of inputs? Uncertainty is a fact about the model in the borader universe? (needs objective knowledge of ground truth to quantify)? And model uncertainty is the estimate of uncertainty in-model? Ty!
-
Noise is information that cannot be compressed. Compression means that you discover structure that makes part of the pattern predictable, so you don’t need to represent that any more.
- 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.