A model compresses a state space by capturing a set of invariances that predict the variance in the states. Its free parameters define the latent space of the model and should ideally fully correspond to the variability, the not-invariant (= unexplained) remainder of the state.
-
Show this thread
-
The proper representation of invariances (which include allowed set of values of the free parameters) is conditional on the reason for an invariance. The relationship to this reason is itself an invariance that needs to be made conditional on the foundations of its semantics.
5 replies 1 retweet 1 likeShow this thread -
Replying to @Plinz
Ok, I'm not smart enough for this thread, but I want to understand! Could you give an example? Sorry if this is a dumb request.
1 reply 0 retweets 0 likes -
Replying to @mattgiammarino
Discovered Invariance in the data = structure of the model (a set of variables with value ranges and a set of computational relationships between them) Discovered Variance in the data = the set of values of the variables that will explain most of the observations I am making
3 replies 0 retweets 0 likes -
Replying to @Plinz
Thank you! That is very helpful. I'm going to chew on it for a bit to ensure my follow up is cogent....
1 reply 0 retweets 0 likes -
Replying to @mattgiammarino
Let me see if I can come up with an example. You make an invariant map of your body surface by counting how often sensory nerves fire simultaneously: these will often be neighbors. Then you can infer when objects are moving over your skin, and reduce the firing of nerves to that.
3 replies 0 retweets 0 likes -
Replying to @Plinz
I can't figure out what 'noise' and 'uncertainty' are . Maybe noise is sensory firing patterns that can't be translated into body locations (using the current function)? uncertainty would be patterns of nerve firing that create ambiguity in the body map? Corrections appreciated
1 reply 0 retweets 0 likes -
Replying to @mattgiammarino
Noise is incompressible information, i.e. information that you cannot relate to other information, such as nerves randomly firing. Uncertainty is estimated difference between model and ground truth. This includes uncertainty about the model quality (i.e. the model of your model).
1 reply 0 retweets 0 likes -
Replying to @Plinz
So noise is where the function cannot resolve particular patterns of inputs? Uncertainty is a fact about the model in the borader universe? (needs objective knowledge of ground truth to quantify)? And model uncertainty is the estimate of uncertainty in-model? Ty!
1 reply 0 retweets 0 likes
Noise is information that cannot be compressed. Compression means that you discover structure that makes part of the pattern predictable, so you don’t need to represent that any more.
-
-
Replying to @Plinz
It's hard to describe how much I love this tweet! It's been making my brain zing off in a million directions.....and having nostalgic memories of the first time my computer uncompressed a video file - seemed like magic at the time. it's still mind bending to my non CS mind!
0 replies 0 retweets 1 likeThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.