Kevin Simler

“Why do you spend so much time fixing bugs instead of concentrating on all the times the software works normally?”

Perhaps a better analogy: why are you worried about how the boiler behaves if it gets overpressurized? Why not just focus on how well it runs the rest of the time?

I don't really get this analogy. As I understand David, he's saying that rationalism focuses on how to maximize EV at explicit decision points, but ignores how to deal with facets of the system that don't show up as explicit choices

It's not [rational = bugs] vs. [metarational = working smoothly], but rather [rational = choice points] vs. [metarational = total behavior of the system]

FWIW, the main criticism of metarationalism that resonates with me is: anything that meta-R points out can then (by virtue of being pointed out) get folded neatly into R. Rationalists absolutely care about all the things metarationalists care about! So where's the meta??

I’m using “rational” to mean roughly “technical” or “formal” rationality specifically, and not jus “good thinking.” (This choice of usage may cause confusion although it’s reasonably standard and I stress the specificity frequently.)

haha yeah I'll bet this causes a lot of problems, because Eliezer strongly branded "rationality" as "good/broad/holistic thinking" over at LW. I'm constantly (mis?)reading your stuff as, "This is what meta-rationalism tries to do _as opposed to_ LW-style rationalism."

Well, he and others of the Berkeley Rationalist community leaders have recently said they regret choosing the word “rationality” because it does have multiple senses and so causes confusion. I’d like to find another word myself, for the same reason, but can’t find one.