Goodhart's Law is just the folk-decision-science version of overfitting, right? Would I lose anything by getting rid of the concept?
-
-
Replying to @regretmaximizer
@___jsf overfitting applies to induction specifically, not really straightforward to generalize.
1 reply 0 retweets 0 likes -
Replying to @ProofOfLogic
@___jsf a remark about overfitting which doesn't generalize: "you typically need an exponential amount of data to support a model's size"
1 reply 0 retweets 1 like -
Replying to @ProofOfLogic
@___jsf (what are "data points" for general Goodhart's Law?)
1 reply 0 retweets 0 likes -
Replying to @ProofOfLogic
@___jsf Also, Goodhart's Law deals with agents, unlike overfitting. Game theory, exploitable vs unexploitable policies can come into it.
1 reply 0 retweets 0 likes -
Replying to @ProofOfLogic
@ProofOfLogic in the analogy I'm imagining, data points are # of observations (however those get defined) that went into making a rule1 reply 0 retweets 1 like -
Replying to @regretmaximizer
@ProofOfLogic I don't see why exponential data size requirements shouldn't get to apply here too, though it would be nice to make an example2 replies 0 retweets 1 like -
Replying to @regretmaximizer
@___jsf I agree it might, just not clear how.
1 reply 0 retweets 0 likes -
Replying to @ProofOfLogic
@ProofOfLogic I think I got generalized "overfitting" -- Goodhart's Law probably has more to do with poor choice of objective function1 reply 0 retweets 0 likes -
Replying to @regretmaximizer
@___jsf I am not sure about that characterization. Take the example of hiring decisions. X can be great criteria until you implement.
2 replies 0 retweets 0 likes
@___jsf It takes more than Bayes Law to correct for people gaming your hiring criteria. Need things difficult to game.
-
-
Replying to @ProofOfLogic
@___jsf If that's what you meant by "poor choice" then ok
1 reply 0 retweets 0 likes -
Replying to @ProofOfLogic
@ProofOfLogic yeah, that prob qualifies as a poor choice. I feel like someone must have written about this at some point...1 reply 0 retweets 1 like - 2 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.