Gregory's Server
Nobody in a position to do anything is in a position to observe the signs something is wrong, and nobody in a position to observe the signs something is wrong is in a position to do something about it. The power differential guarantees that the people with the ability to do something will not listen to the people who have a problem to report. Indeed, they may have made it literally impossible for the people with the observations to contact them in any way.
I mean, if you thought you found a bug in a job application platform while applying for a job, how - where - would you even go about filling a bug report?
And all this is true even before introducing inscrutable, undebuggable AI to the platform. There is no way in which adding AI will make this better.
There's lots of places in our society in which computers lie like walls between two populations divided by a power imbalance. Those computer systems' faults are already largely undetectable for entirely social reasons.
5/?
(I've already written at length about how this phenomenon is fucking up medical records and consequently medical science. Series starts here https://siderea.dreamwidth.org/1540620.html , particularly see part 2 "Power's Deformation of Data".)
Bugs and other faults in such software are like mold growing in crevasses that sunlight can't reach, and which stagnant water never wholly evaporates.
So we can either assume that in these systems in which it's hard-to-impossible to detect faults, there are none to detect, or we have to assume there's probably undetected faults. Probably many, because they will accumulate.
Into these irremediable reservoirs of pestilential technological suckitude, AI is going to be - and presumably already has - been injected.
So that will be just ducky.
And that's why I say I suspect there's already systems screwing things up for everyone already.
7/?
(I've already written at length about how this phenomenon is fucking up medical records and consequently medical science. Series starts here https://siderea.dreamwidth.org/1540620.html , particularly see part 2 "Power's Deformation of Data".)
Bugs and other faults in such software are like mold growing in crevasses that sunlight can't reach, and which stagnant water never wholly evaporates.