Gregory's Server
I've worked on other sorts of enterprise class software. I have a reasonable sense of what kind of faults are harder and easier to debug, or even detect in the first place.
And I am telling you, if such an application somehow got into a state where any resume with the word "Java" on it got filed to /dev/null, there is a zero percent chance anybody would ever figure it out.
"All these applicants are crap! The ad clearly states 5+ years Java experience, but not one applicant even has *any* Java experience! Can nobody read?! Damn it, the FAANGs must have hired all the Java developers."
The reason this kind of bug will never be found is that there's such a steep power differential between, first, the developers of the software and their customers, and then again between the operator-users of the software (the hiring company) and the end-users (the job applicants).
4/?
Nobody in a position to do anything is in a position to observe the signs something is wrong, and nobody in a position to observe the signs something is wrong is in a position to do something about it. The power differential guarantees that the people with the ability to do something will not listen to the people who have a problem to report. Indeed, they may have made it literally impossible for the people with the observations to contact them in any way.
I mean, if you thought you found a bug in a job application platform while applying for a job, how - where - would you even go about filling a bug report?
And all this is true even before introducing inscrutable, undebuggable AI to the platform. There is no way in which adding AI will make this better.
There's lots of places in our society in which computers lie like walls between two populations divided by a power imbalance. Those computer systems' faults are already largely undetectable for entirely social reasons.
5/?
Nobody in a position to do anything is in a position to observe the signs something is wrong, and nobody in a position to observe the signs something is wrong is in a position to do something about it. The power differential guarantees that the people with the ability to do something will not listen to the people who have a problem to report. Indeed, they may have made it literally impossible for the people with the observations to contact them in any way.