🤔 conjecture: x-risk memespace makes the mistake of conflating whether a given factor (namely p(everyone dies))
(1) should usually be more highly weighted in policy DECISIONS
(2) should be a core part of your personal ONTOLOGY for thinking about and framing everything
Conversation
the under-weighting of (1) seems somewhat obvious to me at the level of e.g. government policy in nuclear, bio, etc. where there's minimal evidence of enough weight on "but could this lead to a terminally gigantic accident?"
1
8
but—as with "value ethics is a good way to maximize utilitarian criteria"—I do basically think there's some other way to organize one's mind and attention, that mostly doesn't *center on* x-risk, that would be net better *for x-risk* if generalized to everyone
4
14
i think the problem is it never will be generalized to everyone
1
2
people give their all to mitigating x-risk—not to imply this is wise—because they feel that others won't
1
1
but wait, this is exactly what I'm talking about
"giving your all to stair-building" does NOT actually look like "framing every decision in terms of the stairs-best choice" in practice
Quote Tweet
Replying to @uncatherio @Kirsten3531 and @imperialauditor
so I was sort of just gesturing at, like, if you're building stairs, you need to know things about types of screws and types of saws and stuff, but you won't be asking like "what's the stairs-best type of screw to buy" so much as "what's a cheap wood screw that is 2-3" long?"
2
3
I'm not talking about total weight on x-risk in the final decision, that can (should!) be quite high
I'm talking about working reference frame
"giving your all to x-risk" is has many downsides *as a working frame*
I conjecture is not even the best frame *for x-risk*!!
1
4
yeah, it is hard to think in a way that leads to progress and good decisions if you are constantly returning back to "am i being Good™? am i Doing The Altruism"
1
2
actually a quite close analogy can be made to e.g. flirting - constantly worrying about whether you’re doing flirting right in fact not conducive to flirting! the surprising thing is that this isn’t just due to the short timescale
maybe in general it is mentally healthy to rely a certain amount on intuition, and too much heavily conscious, analytical, "system 2" thinking leads to subtle stressors that build over time
or it could be both of these things
2
1
> too much heavily conscious, analytical, "system 2" thinking leads to subtle stressors that build over time
this for sure is true IME IMO
2