@mcc In my opinion, the real problem with this and other such scenarios really boils down to: just because a "rationalist" can imagine it doesn't mean it's logically consistent with reality.

The most obvious case of this is whenever EY goes on (and on and on, at length) about Newcomb's paradox, which is only possible if you assume that hyperintelligent omniscient beings which can solve the Halting Problem in O(1) time are possible. So much of the Less Wrong dreck falls apart if you know... like... anything about the subjects that EY thinks he's talking about.

In a nutshell, Less Wrong is what happens if you learn everything you know about math, physics, ethics, and philosophy from economists.