@isomeme robots.txt++++. Much more detailed, and on a per-post basis, not site-wide.

And ideally, to avoid a DNT-style let’s-just-ignore-it disaster, ideally the whole thing would be subject to a Ricardian contract framework that anchors the post markup in contract law.