Hadi PartoviVerified account

Humans are astonishingly bad at setting goals. We consistently establish targets that invite manipulation. THREAD: A mental model on goal setting and unintended consequences:

Goodhart’s Law is a simple mental model: When a measure becomes a target, it ceases to be a good measure. If a measure of performance becomes a stated goal, humans tend to optimize for it, regardless of any associated consequences. The measure loses its value as a measure!

Goodhart’s Law is named after British economist Charles Goodhart, who referenced the concept in a 1975 article on British monetary policy. Goodhart wrote that, “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

But the concept was popularized by anthropologist Marilyn Strathern. In a 1997 paper, she generalized the thinking and called it Goodhart’s Law. “When a measure becomes a target, it ceases to be a good measure.” It became a mental model with considerable practical relevance...

The phenomenon has been observed throughout history. Humans consistently establish targets that invite manipulation. Indian Cobras & Soviet Nails. Let’s quickly cover each situation:

The Cobra Effect There were too many cobras in India, so the British colonists started offering bounties for cobra heads. Locals began breeding cobras, killing them, and turning in the heads to earn bounties. Many cobras were released, increasing the population of cobras.

The British measure of cobra elimination (cobra heads) became an explicit target, and thus ceased to be a good measure. Humans optimized for a goal, in spite of the clear negative consequences. (I wrote about The Cobra Effect at length in the thread below!)https://twitter.com/SahilBloom/status/1318926201772101632 …

Soviet Nails Soviet factories set goals based on the number of nails produced. The workers produced thousands of tiny nails. The factories changed to goals based on the weight of nails produced. The workers produced a few massive nails. In both cases, the nails were useless.