Member Of Species

@MemberOfSpecies Hmm, not sure it was clear I was using “∞” as shorthand for “very large, unspecified, finite number.”

@MemberOfSpecies If that *was* clear, I’m not sure how what I attributed to singularitarianism differs from the actual argument.

@Meaningness The difference is in the ε, which I take to refer only to probabilities so small they make max-EU heuristics problematic.

@MemberOfSpecies The problem is more that U and P are both so uncertain that UxP could be anything; was subject of y aborted LW sequence.

@Meaningness The P is very hard to give precisely justified estimates for, but it can be constrained somewhat with informal arguments.

@Meaningness Maybe action in advance of AI progress is only 10% as effective, and, say, $100M buys you 0.05% instead of 0.5% risk reduction.

@Meaningness Saying, "yes, this argument decreases how much success we should anticipate, but the payoff is huge" seems reasonable here.

@MemberOfSpecies Agreed. But then, the question is, can you do anything with $10M *at all*, in absence of plausible starting points?