|
Top-level
 What do we do about this? One answer would be to force the platforms to provide service to users who, in their judgment, might be engaged in fraud, or trafficking in CSAM, or arranging terrorist attacks. This is not my preferred solution, for reasons that I hope are obvious! We can try to improve the decision-making processes at these giant platforms so that they catch fewer dolphins in their tuna-nets. 13/ 30 comments
This hasn't worked very well. A proposal for "second wave" moderation oversight based on arms-length semi-employees at the platforms who gather and report statistics on moderation calls and complaints hasn't gelled either: https://pluralistic.net/2022/03/12/move-slow-and-fix-things/#second-wave 15/ Both the EU and California have privacy rules that allow users to demand their data back from platforms, but neither has proven very useful (yet) in situations where users have their accounts terminated because they are accused of committing gross violations of platform policy. 16/ You can see why this would be: if someone is accused of trafficking in child porn or running a pig-butchering scam, it would be perverse to shut down their account but give them all the data they need to go one committing these crimes elsewhere. But even where you *can* invoke the EU's GDPR or California's CCPA to get your data, the platforms deliver that data in the most useless, complex blobs imaginable. 17/ For example, I recently used the CCPA to force Mailchimp to give me all the data they held on me. Mailchimp - a division of the monopolist and serial fraudster Intuit - is a favored platform for spammers, and I have been added to *thousands* of Mailchimp lists that bombard me with unsolicited press pitches and come-ons for scam products. 18/ Mailchimp has spent a decade ignoring calls to allow users to see what mailing lists they've been added to, as a prelude to mass unsubscribing from those lists (for Mailchimp, the fact that spammers can pay it to send spam that users can't easily opt out of is a feature, not a bug). 19/ I thought that the CCPA might finally let me see the lists I'm on, but instead, Mailchimp sent me more than 5900 files, scattered through which were the internal serial numbers of the lists my name had been added to - but without the names of those lists any contact information for their owners. I can see that I'm on more than 1,000 mailing lists, but I can't do anything about it. 20/ Mailchimp shows how a rule requiring platforms to furnish data-dumps can be easily subverted, and its conduct goes a long way to explaining why a decade of EU policy requiring these dumps has failed to make a dent in the market power of the Big Tech platforms. 21/ The EU has a new solution to this problem. With its 2024 Digital Markets Act, the EU is requiring platforms to furnish APIs - programmatic ways for rivals to connect to their services. With the DMA, we might finally get something parallel to the mobile "number portability" for other kinds of platforms. If you've ever changed cellular platforms, you know how smooth this can be. When you get sick of your carrier, you set up an account with a new one and get a one-time code. 22/ Then you call your old carrier, endure their pathetic begging not to switch, give them that number and within a short time (sometimes only minutes), your phone is now on the new carrier's network, with your old phone-number intact. This is a much better answer than forcing platforms to provide service to users whom they judge to be criminals or otherwise undesirable, but the platforms *hate* it. 23/ They *say* they hate it because it makes them complicit in crimes ("if we have to let accused fraudsters transfer their address books to rival services, we abet fraud"), but it's obvious their objection is *really* about being forced to reduce the pain of switching to a rival. There's a superficial reasonableness to the platforms' position, but only until you think about Mark, or K Renee, or other people who've been "unpersonned" by platforms with no explanation or appeal. 24/ The platforms have rigged things so that you *must* have an account with them in order to function, but they also want to have the unilateral right to kick people off their systems. The combination of these demands represents more power than *any* company should have, and Big Tech has repeatedly demonstrated its unfitness to wield this kind of power. 25/ This week, I lost an argument with my accountants about this. They provide me with my tax forms as links to a Microsoft Cloud file, and I need to have a Microsoft login in order to retrieve these files. This policy - and a prohibition on sending customer files as email attachments - came from their IT team, and it was in response to a requirement imposed by their insurer. 26/ The problem here isn't merely that I must now enter into a contractual arrangement with Microsoft in order to do my taxes. It isn't just that Microsoft's terms of service are ghastly. It's not even that they could change those terms at any time, for example, to ingest my sensitive tax documents in order to train a large language model. 27/ It's that Microsoft - like Google, Apple, Facebook and the other giants - routinely disconnects users for reasons it refuses to explain, and offers no meaningful appeal. Microsoft tells its business customers, "force your clients to get a Microsoft account in order to maintain communications security" but also reserves the right to unilaterally ban those clients from having a Microsoft account. 28/ There are examples of this all over. Google recently flipped a switch so that you can't complete a Google Form without being logged into a Google account. Now, my ability to purse all kinds of matters both consequential and trivial turn on Google's good graces, which can change suddenly and arbitrarily. 29/ If I was like Mark, permanently banned from Google, I wouldn't have been able to complete Google Forms this week telling a conference organizer what sized t-shirt I wear, but also telling a friend that I could attend their wedding. Now, perhaps some people *really* should be locked out of digital life. Maybe people who traffick in CSAM should be locked out of the cloud. 30/ But the entity that should make that determination is a court, not a Big Tech content moderator. It's fine for a platform to decide it doesn't want your business - but it shouldn't be up to the platform to decide that *no one* should be able to provide you with service. This is especially salient in light of the chaos caused by Crowdstrike's catastrophic software update last week. 31/ Crowdstrike demonstrated what happens to users when a cloud provide *accidentally* terminates their account, but while we're thinking about reducing the likelihood of such accidents, we should really be thinking about what happens when you get Crowdstruck *on purpose*. The wholesale chaos that Windows users and their clients, employees, users and stakeholders underwent last week could have been pieced out retail. 32/ It could have come as a court order (either by a US court or a foreign court) to disconnect a user and/or brick their computer. It could have come as an insider attack, undertaken by a vengeful employee, or one who was on the take from criminals or a foreign government. The ability to give anyone in the world a Blue Screen of Death could be a *feature* and not a bug. 33/ It's not that companies are sadistic. When they mistreat us, it's nothing personal. They've just calculated that it would cost them more to run a good process than our business is worth to them. If they know we can't leave for a competitor, if they know we can't sue them, if they know that a tech rival can't give us a tool to get our data out of their silos, then the expected cost of mistreating us goes down. 34/ That makes it economically rational to seek out ever-more trivial sources of income that impose ever-more miserable conditions on us. When we can't leave without paying a very steep price, there's practically a fiduciary duty to find ways to upcharge, downgrade, scam, screw and enshittify us, *right up to the point* where we're so pissed that we quit. 35/ Google could pay competent decision-makers to review every complaint about an account disconnection, but the cost of employing that large, skilled workforce vastly exceeds their expected lifetime revenue from a user like Mark. The fact that this results in the ruination of Mark's life isn't Google's problem - it's Mark's problem. 36/ The cloud is many things, but most of all, it's a trap. When software is delivered as a service, when your data and the programs you use to read and write it live on computers that you don't control, your switching costs skyrocket. Think of Adobe, which no longer lets you buy programs at all, but instead insists that you run its software via the cloud. Adobe used the fact that you no longer own the tools you rely upon to cancel its Pantone color-matching license. 37/ They are designed to shift power away from users to software companies. An app is just a web-page wrapped in enough IP to make it a felony to add an ad-blocker to it. A cloud app is some Javascript wrapped in enough terms of service clickthroughs to make it a felony to restore old features that the company now wants to upcharge you for. 39/ Google's defenstration of K Renee, Mark and Cassio may have been accidental, but Google's *capacity* to defenstrate *all of us*, and the enormous cost we all bear if Google does so, has been carefully engineered into the system. Same goes for Apple, Microsoft, Adobe and anyone else who traps us in their silos. 40/ The lesson of the Crowdstrike catastrophe isn't merely that our IT systems are brittle and riddled with single points of failure: it's that these failure-points can be tripped *deliberately*, and that doing so could be in a company's best interests, no matter how devastating it would be to you or me. 41/ Support me this summer on the Clarion Write-A-Thon and help raise money for the Clarion Science Fiction and Fantasy Writers' Workshop! https://clarionwriteathon.com/members/profile.php?writerid=293388 42/ Image: CC BY 3.0 eof/ |
Gregory's Server
The "first wave" of content moderation appeals focused on the establishment of oversight and review boards that wronged users could appeal their cases to. The idea was to establish these "paradigm cases" that would clarify the tricky aspects of content moderation decisions, like whether uploading a Nazi atrocity video in order to criticize it violated a rule against showing gore, Nazi paraphernalia, etc.
14/