@andybaio Pretty much. Mostly because the idea of a consent mode (much outside of authentication and respecting robots.txt) is something imposed upon the tech sector, not generally something it grew from within.

A lot of people pulling data from the internet are doing so under the assumption that the data is there because somebody wants it pulled and they are surprised when they are post-hoc told no.