Email or username:

Password:

Forgot your password?
Top-level
Emily M. Bender (she/her)

Back to the Reuters article. What's worse than reporting on non-peer-reviewed, poorly written, "research" papers posted to the web? Reporting on vague descriptions of a "discovery" attributed only unnamed sources. 11/

What's their evidence that there's a big breakthrough? Something that has "vast computing resources" can do grade-school level math. You know what else can do grade-school level math? A fucking calculator that can run on a tiny solar cell. Way more reliably, too, undoubtedly. 12/

Screencap: 

Some at OpenAI believe Q* (pronounced Q-Star) could be a breakthrough in the startup's search for what's known as artificial general intelligence (AGI), one of the people told Reuters. OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks.

Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because the individual was not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said.
31 comments
Emily M. Bender (she/her)

Could not verify, eh? And yet decided it was worth reporting on? Hmm... 13/

Screencap:

Reuters could not independently verify the capabilities of Q* claimed by the researchers.
Emily M. Bender (she/her)

"AI" is not "good at writing"—it's designed to produce plausible sounding synthetic text. Writing is an activity that people to do as we work to refine our ideas and share them with others. LLMs don't have ideas. 14/

(And it bears repeating: If their output seems to make sense, it's because we make sense of it.) 15/

Screencap:

Researchers consider math to be a frontier of generative AI development. Currently, generative AI is good at writing and language translation by statistically predicting the next word, and answers to the same question can vary widely. But conquering the ability to do math — where there is only one right answer — implies AI would have greater reasoning capabilities resembling human intelligence. This could be applied to novel scientific research, for instance, AI researchers believe.
Emily M. Bender (she/her)

Also, it's kind of hilarious (lolsob) that OpenAI is burning enormous amounts of energy to take machines designed to perform calculations precisely to make them output text that mimics imprecisely the performance of calculations ... and then deciding that *that* is intelligent. 16/

But here is where the reporting really goes off the rails. AGI is not a thing. It doesn't exist. Therefore, it can't do anything, no matter what the AI cultists say. 17/

Screencap:

Unlike a calculator that can solve a limited number of operations, AGI can generalize, learn and comprehend.
Emily M. Bender (she/her)

And before anyone asks me to prove that AGI doesn't exist: The burden of proof lies with those making the extraorindary claims. "Slightly conscious (if you squint)", "can generalize, learn and comprehend" are extraordinary claims requiring extraordinary evidence, scrutinized by peer review. 18/

Emily M. Bender (she/her) replied to Emily M. Bender (she/her)

Next stop: both-sides-ing reporting of "existential risk". OpenAI is deep within the TESCERAList cult. It's staffed by people who actually believe they're creating autonomous thinking machines, that humans might merge with one day, live as uploaded simulations, etc. 19/

Screencap:

In their letter to the board, researchers flagged AI’s prowess and potential danger, the sources said without specifying the exact safety concerns noted in the letter. There has long been discussion among computer scientists about the danger posed by highly intelligent machines, for instance if they might decide that the destruction of humanity was in their interest.
Emily M. Bender (she/her) replied to Emily M. Bender (she/her)

It is an enormous disservice to the public to report on this as if it were a "debate" rather than a disruption of science by billionaires throwing money at the hope of bringing about the speculative fiction stories they grew up reading---and philosophers and others feeling important by dressing these same silly ideas up in fancy words. 20, 21/

Emily M. Bender (she/her) replied to Emily M. Bender (she/her)

If TESCREAL as an acronym is unfamiliar, start with this excellent talk by @timnitGebru , reporting on joint work with @xriskology connecting the dots: 22/

youtube.com/watch?v=P7XT4TWLzJ

Emily M. Bender (she/her) replied to Emily M. Bender (she/her)

The article ends as it began, by platforming completely unsubstantiated claims (marketing), this time sourced to Altman:

23/

Screencap:

In addition to announcing a slew of new tools in a demonstration this month, Altman last week teased at a summit of world leaders in San Francisco that he believed major advances were in sight.

"Four times now in the history of OpenAI, the most recent time was just in the last couple weeks, I've gotten to be in the room, when we sort of push the veil of ignorance back and the frontier of discovery forward, and getting to do that is the professional honor of a lifetime," he said at the Asia-Pacific Economic Cooperation summit.

A day later, the board fired Altman.
Emily M. Bender (she/her) replied to Emily M. Bender (she/her)

To any journalists reading this: It is essential that you bring a heavy dose of skepticism to all claims by people working on "AI". Just because they're using a lot of computer power/understand advanced math/failed up into large amounts of VC money doesn't mean their claims can't and shouldn't be challenged. 24/

Emily M. Bender (she/her) replied to Emily M. Bender (she/her)

There are important stories to be reporting in this space. When automated systems are being used, who is being left without recourse to challenge decisions? Whose data is being stolen? Whose labor is being exploited? How is mass surveillance being extended and normalized? What are the impacts to the natural environment and information ecosystem? 25/

Emily M. Bender (she/her) replied to Emily M. Bender (she/her)

Please don't get distracted by the dazzling "existential risk" hype. If you want to be entertained by science fiction, read a good book or head to the cinema. And then please come back to work and focus on the real world harms and hold companies and governments accountable. /fin

Neia replied to Emily M. Bender (she/her)

@emilymbender@dair-community.social Yeeeep. If we're looking for an analogy with fiction, it's less Skynet, more of a digital WALL-E.

Frank Bennett replied to Emily M. Bender (she/her)

@emilymbender Years ago, I set up LDA and ran some jobs through it in preparation for a law review article that I never got around to completing. At that time there were two other pieces out there that made assertions about law based on its output—factual, conclusive claims, despite the *developer* of the system (David Bliss, IIRC) clearly stating that it only produced statistical correlations based on pattern matching, so you shouldn't do that. The AI hype is through-the-looking-glass deja vu.

mav :happy_blob: replied to Emily M. Bender (she/her)

@emilymbender
Are there any systems left by which to actually hold anyone accountable, though? That's the part of this that terrifies me: tens of billions of dollars and who knows how many human hours of research being done by an unaccountable company for surely negative ends, and humankind has given up on placing any controls on capitalism that actually do anything.

If they do ever invent AGI, there's no possible positive outcome.

Nicole Parsons replied to mav

@mav @emilymbender

Examine the sources of the funding for this hyped up concept of AI.
Despots. Oil oligarchs. Mentally ill tech lords. Kleptocrats. Seditious GOP donors.

It's the same tax-evading billionaires behind frauds like cryptocurrency, carbon offsets, & NFT's - the "something for nothing" conmen

Mass tech layoffs to undermine content moderation. Those layoffs were ordered by the investors

Buried in the hype, is the intent to launch AI-driven anti-democracy disinformation campaigns

@mav @emilymbender

Examine the sources of the funding for this hyped up concept of AI.
Despots. Oil oligarchs. Mentally ill tech lords. Kleptocrats. Seditious GOP donors.

It's the same tax-evading billionaires behind frauds like cryptocurrency, carbon offsets, & NFT's - the "something for nothing" conmen

Nicole Parsons replied to Nicole

@mav @emilymbender

AI is replacing "algorithmic amplification" as the plausible deniability excuse for the 2024 election cycle.

Investors in AI:
Founders Elon Musk, Greg Brockman, Ilya Sutskever, John Schulman, Sam Altman, Wojciech Zaremba
crunchbase.com/organization/op
en.m.wikipedia.org/wiki/OpenAI

Reminder: JPMorganChase orchestrated the loans for Musk's purchase of Twitter.

None of these people want democracy to survive. Their oil investors certainly don't.
Lawrence Summers
Peter Thiel
Infosys

@mav @emilymbender

AI is replacing "algorithmic amplification" as the plausible deniability excuse for the 2024 election cycle.

Investors in AI:
Founders Elon Musk, Greg Brockman, Ilya Sutskever, John Schulman, Sam Altman, Wojciech Zaremba
crunchbase.com/organization/op
en.m.wikipedia.org/wiki/OpenAI

Heskie replied to mav

@mav @emilymbender
I have mentioned this before, but it is relevant to repeat here, the words of AC Grayling
"Anything that CAN be done WILL be done if it brings advantage or profit to those who can do it." and
"What CAN be done will NOT be done if it brings costs, economic or otherwise, to those who can stop it"
It is much more (currently) relevant to autonomous weapons systems. More: thearticle.com/graylings-law

Dieu replied to Emily M. Bender (she/her)

@emilymbender Greg Bear may be is a nice read for people into doom and simulated humans.

Matthew Exon replied to Emily M. Bender (she/her)
@emilymbender The most heartening thing I saw recently was mention of an internal poll of OpenAI employees as to when AGI will be achieved, and the median answer was "15 years". "In 15 years" is a term of art in AI research meaning "approximately never and a half". It suggests the people actually building the tools have their heads screwed on moderately securely.
Mina replied to Emily M. Bender (she/her)

@emilymbender

All this ai soap opera is tech bro PR, IMO.

Besides: I don't fear ai. I fear capitalists and governments who intend to put human decisions in inhuman(e) hands.

Emma Jezebel Cat Lady Byrne replied to Emily M. Bender (she/her)

@emilymbender I'm off to give a talk at a business event today where I'll be holding this line. Sometimes I feel like Cassandra...

Thank you for keeping on keeping on in the face of journalists, politicians and business people losing their minds over imaginary threats while the voices of those suffering now are ignored

Sophie Schmieg replied to Emily M. Bender (she/her)

@emilymbender imagine being trained on the near totality of humanity's knowledge, and struggling to perform grade school mathematics.

We build accidental calculators all the time, if anything it's remarkable how much this approach struggles with being one.

DELETED replied to Emily M. Bender (she/her)

@emilymbender A wonderful thread. This post especially resonates today. Thank you!

François Galea replied to Emily M. Bender (she/her)

@emilymbender There could be a problem with peer review IMO. In the "AI" research field, you can find researchers with the AI hype bias who can positively review those extraordinary claims even without strong evidence.
Or am I being too pessimistic ?

Nat

@emilymbender “generalize, learn, and comprehend” is, hilariously, SO CLOSE to the same phrasing used in the 1958 breathless reporting on the invention of the perceptron in the New York Times article entitled “electronic brain teaches itself”. incredible that these guys’ predecessors talked about the state of the art at that time the same way.

nytimes.com/1958/07/13/archive

🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦

@emilymbender I'm not a technical person, but I do work in marketing. I can smell dark marketing techniques a mile away, and the whole "AI" realm reeks of it from halfway around the world.

I made an "AI" write a position paper on the value of Quantum Chromodynamics in marketing: gamerplus.org/notes/9icnhgaalz

I made an "AI" generate a picture with very specific instructions that it completely failed to follow (attached, with prompt).

It is trivial to expose these "AIs" for what they really are.

@emilymbender I'm not a technical person, but I do work in marketing. I can smell dark marketing techniques a mile away, and the whole "AI" realm reeks of it from halfway around the world.

I made an "AI" write a position paper on the value of Quantum Chromodynamics in marketing: gamerplus.org/notes/9icnhgaalz

Drew Mochak

@emilymbender LOL they didn't even see the letter. This whole thing is some of the shoddiest reporting I've seen in a while.

Sara

@emilymbender I saw this article yesterday and thought, well hell, I can perform mathematical operations at a grade school level and the only resources I require to do so are sandwiches and coffee and maybe a pencil

Jennifer Kayla | Theogrin 🦊

@emilymbender
Thank you for this excellent thread!

This style of reporting, breathlessly mimicking the words of the People That (should) Know, always strikes me as fundamentally incurious. LLMs are complex-seeming beasts, ponderous and burdensome, but peeling away the artifice -- and this is something journalists need to relearn how to do, it seems! -- reveals a tool which doesn't even do grade-school math, it just pattern-matches and says, 'this is probably the answer for which you're looking' with no logic involved.

Which, I suppose, is in some small ways not dissimilar to a journalist who gets fed garbage, puts up a puff piece about Sam Altman's 'triumphant return', and expects that it's what the readers and financiers are looking for. No logic or proof necessary, or for that matter actual reporting.

@emilymbender
Thank you for this excellent thread!

This style of reporting, breathlessly mimicking the words of the People That (should) Know, always strikes me as fundamentally incurious. LLMs are complex-seeming beasts, ponderous and burdensome, but peeling away the artifice -- and this is something journalists need to relearn how to do, it seems! -- reveals a tool which doesn't even do grade-school math, it just pattern-matches and says, 'this is probably the answer for which you're looking' with...

Go Up