@baldur It’s almost like it can’t be trusted to do anything useful and it’s just a toy that is slowly melting the planet…

@baldur "the health system complies with state and federal privacy laws."

Case closed. Don't worry about it.

@baldur what could possibly go wrong.

@baldur
This can only end well.

@baldur@toot.cafe

First a failure to perfect dumb Voice Recognition, refuse to pay Stenographers and now this.
Add to ADM...

Hospitals invent enough by themselves as it is.

@baldur well no one saw that coming, did they?
My wife used to work in medical transcription - you CANNOT trust this job to AI, it will never, ever work.

@baldur Literally insane. That is the purpose of a LLM: to make stuff up. That's the PURPOSE. That's what it does. There are hundreds of voice transcription tools that work well, why would anyone ever want one that makes stuff up?

@baldur "AI" hype is certainly eye opening regarding how far well-funded PR cab push obviously bad things into public discourse. I am forced to look askance at people who still uncritically post positive news about ChatGPT.

@baldur Five years from now people will think we were nuts to place so much public trust in and corporate money on unreliable AI.

@baldur
What I don't get is, audio transcription is a solved problem. We have software that does a pretty good job, with reasonable energy and resource use and basic human supervision; my partner used to have a job proofreading auto-transcriptions.
Best part is, when it can't make something out, the legit transcription software goes "{UNINTELLIGIBLE}" and the human proofreader fixes it. It never invents stuff.
We don't need a way to do it worse with bigger resource use and sneaky errors.

@baldur

Didn't they test it before using it?

@baldur [In an example they uncovered, a speaker said, “He, the boy, was going to, I’m not sure exactly, take the umbrella.”

But the transcription software added: “He took a big piece of a cross, a teeny, small piece ... I’m sure he didn’t have a terror knife so he killed a number of people.”]

"Take the umbrella" to "terror knife so he killed a number of people" is a hell of a mistranscription

And this is used to transcribe medical notes !?

And it deletes the recording "for privacy" riiight🤦‍♀️

@baldur
But what exactly ? Maybe it shows how conformity is kidding us. we beed to stay creative... But it's not the best way to be in charge...

@baldur I wish they would stop using the word 'hallucination'. They are spouting garbage. Transcribing voice recordings is hard (I do that at least bi-weekly) and I see what automated systems do for the meetings I run and the output isn't even close to what is being said and some are complete inventions. Technical terms and acronyms 95%+ wrong in the transcriptions generated.

@baldur "he found hallucinations in 8 out of every 10 audio transcriptions he inspected,"
Well then that isn't a transcription tool, it is a #BullshitEngine

@baldur
I've read that (fascinating) piece twice, and I see there are tons of links for the people and institutions mentioned, but the underlying studies they mention will be a real hassle to try and find without links.

@baldur IMO it's something that can easily be noticed by users: I have integrated a distilled French-targeted Whisper model in my day-to-day usage to be able to understand what people tell me in voice messages, and even during tests I could SEE that it was making some stuff up.

I'm not talking about mistakes, like misunderstanding a word for another. Straight up sentences that could appear out of thin air if the speaker sighed too loudly.

Thankfully I don't rely solely on the transcription...

@baldur AI always does and will always do things like this. It must not be used in any critical infrastructure under any circumstances.

@baldur priceless :thisisfine:

@baldur same thing in ordinary business and legal transcription. That’s why we must keep audio recording and transcript.

@baldur Too many vendors' public-facing folks frame this "hallucination" defect of their commercial #ai as a training issue for the subscriber --- while the vendor continues to enrich itself by training its ai on medical work product & #patient health care records.

#hipaa #publichealth #ethics #fedilaw #opensource #opendata

@baldur Of course it did, it only exists to make shit up and poorly at that. Just... look, do some text to speech, get someone human to do a cleanup and consider that good enough. Someone will die from this nonsense if they haven't already.

@baldur We clearly need the Ghostbusters.

@baldur So itll be GREAT for use in healthcare 🤡💩⚰️

@baldur shocked I say

@baldur why is that surprising?

@baldur

I've seen alot of errors when I've read AI. At this point it's not viable

@baldur Using "AI" algorihms for anything unchecked is like thinking Bonaqua is more than a plastic bottle.

@baldur THIS is why #AI should not be allowed...

@baldur I keep telling people it's not real AI, it's real BG. It's a large language model with algorithm engines that take the accumulated wisdom and beauty and art of the entire history of the human race and churns out garbage. (BS Generator)

@baldur "It’s impossible to compare Nabla’s AI-generated transcript to the original recording because Nabla’s tool erases the original audio for 'data safety reasons,' Raison said."

So Nabla is taking the irresponsibility to a new level.

@baldur @inthehands I recently talked with a web developer who had been working on an application for recording biological lab values.

Their group was unaccountably told to try using ChatGPT.

They stopped when it immediately inserted impossible values into the data tables.

@baldur Yes but it's way faster to do it wrong than right and that's the only metric that matters somehow.

@baldur Actually, humans do this as well...

@baldur 🍿 👀

@baldur why do they even need AI to transcribe? To detect who is talking? Is it supposed to improve accuracy? (I'm just musing out loud)

@baldur

My goodness.
I am so shocked by this.

#AI #StopAI

@baldur

The researches should be replaced by AI too!

@baldur

Oh Big Fucking Surprise..!