@david_chisnall Amen!

@david_chisnall

Maybe in the Star Trek world they use an unambiguous logique language, easy to translate in any unambiguous language, but we loose that in the English translation.

That would explain a lot the universal translator ;)

@david_chisnall I think the unfortunate challenge is that some people do not want tools, they want servants or slaves. LLM holds that prospect for them.

What happens when a lowly servant misinterprets the intent of a command? They are blamed for it irrespective of ambiguity.

@david_chisnall I'm not so sure. I often express myself in natural language to ask people to do things, and that usually works out pretty well.

So it's possible in principle, it's just not something that computers can do yet. Maybe one day they will.

@david_chisnall LLMs play directly into business guy's IT wet dreams. If you've ever developed software for business guys, you realize that the biggest problem is figuring out what the business guy wants. Even on those rare occasions when the business guy knows what he wants, his ability to express it in a manner that can be converted into software is limited at best. For these guys, the idea of LLMs magically interpreting their wants and needs, is wonderous.

@david_chisnall Early morning and pre-coffee and my mind went to Luxembourgish which is primarily a Germanic language which added in French to deal with legal and administrative issues. I don’t speak it, but I think this combination is possibly the best way to obfuscate meaning. Otherwise agree with your premise and can only add that language exists in a context. LLMs sorely lack the context for language to be completely understood.

@david_chisnall makes me think of https://xkcd.com/191/

@david_chisnall
> (and not what some other person saying the same thing at the same time might have meant)

see "Pierre Menard, Author of the Quixote" by Borges

@david_chisnall

Arthur C. Clarke had an example of the over-engineering trap. Solve this problem: allow a farmer to direct a draft-horse to turn left or right on command.

1/

@david_chisnall The second of your two languages was the project of the logical positivists. They eventually decided it was not possible.

@david_chisnall @cstross LLMs are truly a dumb person‘s idea of what programming is.

@david_chisnall Natural language interfaces are akin to magic. It's all under the assumption that the intent is somehow, magically recognised. Funnily, there are cautionary tales re magic addressing exactly this aspect. Almost all stories involving wishes turn the phrasing against the one expressing their wishes (see djinns, fairies, etc.).

@david_chisnall what you’re describing is essentially the old problem of “do what I mean, not what I say”.

On top of the ambiguity of natural language a lot of people can’t properly express themselves even when the language isnt ambiguous by itself.

@david_chisnall

Babel, when the Tower Crumbled.
#Darmok #StarTrekTNG

@david_chisnall
Thanks for a new thought to think. Your post for me fell into the ‘obvious once said’ department.

@david_chisnall this aligns with my thinking also.. determinism and logic are critical in any programming interfaces! even a skilled programmer writing their own code with understanding of clear outcomes will still get caught in flawed logic flows in their app. being able to clearly articulate how you want something to work is important, and you dont want it to change based on new datasets learned by the model.

@david_chisnall I think that instead of making systems that use an exacting, very precise, unambiguous language (which really doesn't exist in reality anyway) that you then force everybody in the whole world to learn so they can communicate with computers …

… why not just make computers not so utterly god-damned fragile that the slightest error makes things get broken and hard?

Make your technology antifragile instead of making people adapt to tech. Because the latter is not gonna happen.

@david_chisnall a small point, but re driving or cooking, I recently switched back from Gemini to Google Assistant in that role. Gemini can't actually do anything more in terms of remote control, so the added complexity just dials up the unwanted responses without adding anything at all. A smaller model is best for remote control even if I must learn the interface by figuring out phrases that work and sticking to them.

@david_chisnall dope asf. really really well put. 🏆

@david_chisnall I have a vague recollection of a science fiction story that had a “robot language.” Maybe Asimov?

@david_chisnall

Superb characterization of #AI that, hopefully, relegates its use to well-defined and safe domains of use (call center routing) or as a fun curiosity for bored people. It should never be used for critical decision making, such as “dangerousness” determinations for bail decisions.

#LLM

@david_chisnall the only reason it's done like this in Star Trek is because it's not a crew running a spaceship, it's a bunch of actors in a TV show. And it wouldn't make for very good TV to have people stare at screens and press buttons.

The crew aren't talking to the computers and vice versa, they're talking to the audience so they know what's going on.

It's beyond mad to retrofit this paradigm onto real tech that real people are trying to use to do real work.

@david_chisnall

If it is not cool to use natural language to control things we should stop listening to you.

@david_chisnall
There is a recent game called Chants of Senaar that explores this "use-based language": a city where each caste (priests, soldiers, artists, scientists) have their own language that you need to learn, each language fit for expressing different concepts that were alien to other caste.

@david_chisnall

"Time flies like an arrow."
"Fruitflies like a banana."

@david_chisnall I'm all with you, but it made me think about the nature of natural language ambiguity. It seems it's all/mostly a matter of them *allowing* us to say things without well-defined conditions for their truth etc. Natural language *could* certainly be used to say pretty much anything unambiguously, but that would be a very regimented *use* of natural language resources. And learning to use your language that way would be quite as hard as learning any formal language.

@david_chisnall

Yes, some confusion stems from science fiction equating microprocessors or machines to conscious, captive minds.

Yet the ambiguity of language for some purposes is not the issue: it's the fact that the world is not automatically reducible to a machine representation with a small set of symbols. In the course of permitting language to be used for machine instructions, for the convenience of programmers, you alter its use and meaning in context.

@david_chisnall if legalese were unambiguous, there wouldn’t be so many court cases arguing about ambiguous laws.

@david_chisnall

Also everything that is wrong with Perl.

@david_chisnall I think this relates to something I learned only after delving deep in C after a decade or so of object-oriented, heavily abstracted programming:

One should always strive to solve the problem at hand, not some generalized, abstract set of N problems. The latter always ends up becoming ritualistic hope-magic, that in the future things will be better if we do things now The Right Way. Aesthetics, law and protocol become metrics instead of whether it solves the problem.

@david_chisnall "built entirely out of undefined behavior" - that stings sharp!

@david_chisnall I scrolled through a lot of the replies here, it's unfortunate I don't see accessibility mentioned. You alluded to it with your car example.

I don't think I get why the issue isn't that LLMs can't understand what you're saying, not that speech input can be ambiguous.

"OK google, play XYZ"

Sure, playing XYZ on YouTube music. You will need to install..

"Cancel. Play XYZ on Spotify"

Sure, playing XYZ on Spotify

An actual interaction I've had with my car (Gemini wasn't used)

@david_chisnall ...and even in Star Trek, Jean-Luc Picard *ALWAYS* makes sure to disambiguate his beverage choice: "Tea, Earl Grey, hot".

(Has the replicator control computer served him cold, lukewarm or scalding tea in the past? Has he once ordered Earl Grey and gotten a weird replica of the actual earl?)

@david_chisnall
"In C, compilers do this for undefined behaviour and it is widely regarded as a disaster. LLMs are built entirely out of undefined behaviour".

So AI is "take the worst mistakes from C and discard the rest"🤔🤣🤪🤡

@david_chisnall
Language for humans is problematic.

@david_chisnall
Just a few notes (not to detract)
1. Mathematics uses a surprising amount of natural language in-between equations. There are fields where this is less so but it is rare.
2. Often, jargon (such as professional jargon or legalese) also exists for other purposes, like discrimination or prestige, and unnecessarily reinvents or complicates phrases.
3. Programming languages are often rife with “sharp corners” where slight, unintuitive differences matter a lot.

@david_chisnall Natural language relies on prosody which pretty much by definition is not available in the words. Facial expression, gestures, and tone of voice change the meaning of words and machine learning has no access to this with the possible exception of words that append emojis, and even then the emoji could be used ironically.

@david_chisnall At least once a week at my place:
Me: Siri, lamp on.
[Lamp turns on]
Me: Siri, TV on.
[TV turns on]
Me: Siri, aircon on.
Siri: The aircon is off.
Me: Fuck! Siri, turn aircon on.
[Airconditioner turns on]
😡

Two out of three ain't bad ... I guess 🤷‍♂️

@david_chisnall So the essence is: LLMs should be widely regarded as a disaster. I definitely won't disagree.

@david_chisnall

Natural languages depend on redundancy. It's why "Coca-Cola" becomes "Bite the wax tadpole" when you try to limit yourself to three or four sound groups.

Among a common group sometimes the redundancy is not the words used, but the phrases have a meaning to the group beyond the words.

For an AI to accept natural language would require it to be intelligent enough to consider all outcomes of its' actions and choose the most reasonable one. Asking where unsure.

@david_chisnall The idea of using natural language to control things predates Star Trek in AI research. Natural Language Processing (NLP) research dates to the 1950s. By the time I was studying NLP under Winograd in the 1980s the need for limited unambiguous vocabulary and grammar for command and control (CC) was obvious.

But CC is not the only use of NLP as the speech recognition systems developed in the 90s demonstrated.

Such tools need not be perfect to be useful.

@david_chisnall

Case in point:

I did some simple DIY the other day with a glue called "no more nails"

Then I tried to look up how long it'd be before the glue was cured.

Google bifurcated nicely between answers about the glue I'd used and about artificial finger nails mostly dropping off.

As each article started it was unclear for a line or two which of the two they were talking about :((

@david_chisnall And even the computers in Star Trek have to be spoken to in a particular way to get the results you want. I don't think "Yo, computer, where's my man Data at?" would work instead of "Computer, locate Lieutenant-Commander Data".

I can't remember the SF writer, but it reminds me of a description of someone as having the clear enunciation and precise language of a person who spent a lot of time talking to computers.

@david_chisnall and yet, most human communication is done using natural language. Software developers talk to project managers every single day without using code to communicate what's happening.

The notion that it's not possible to do clear communication without using a formal system doesn't hold water.

There are also lots of solutions that can mitigate the problems you're describing. For example, the system can output the work plan in steps, and you can revise individual steps.

@david_chisnall

Most models trained by Adversarial Neural Networks (i.e. 2 machine learning algorithms working together: 1 suggesting, 1 discriminating the suggestions) have the human, and therefore expensive, limit of well-labelled input data. LLM input data is inherently labelled to perfection.

This is why they are great and a leap that should be relished. Open sourced models keep access to this wonderful technology in the hands of the many and not the few.

And why cars still need drivers

@david_chisnall Star Trek is the source of a lot of ills. For example, most of the public thinks that challenging engineering issues in a hostile environment of radiation and vacuum can be solved in an hour.

@david_chisnall ... and this is why lawyers have jobs. A fact I am profoundly grateful for, given I have bills to pay, but I do sometimes wonder whether there ought not to be a better way than arguing about the significance of a comma.

@david_chisnall

Permanently disabled people who need voice interfaces exist.

@david_chisnall

Computer, fire.

Now is that open fire? Or there's a fire on the bridge?

Getting the correct weather out of Alexa is bad enough.

@david_chisnall
Write instructions so someone else can do one of your jobs, with minimal training.
(It will take a week and a lot of revisions, because what you know that they don't is not obvious.)

@david_chisnall I will save this post for later until I experiment with one approach I have in mind

@david_chisnall

The principles embodied by #LLM (Language) models are actually used in other modalities of AI.

There are voice, image and video modalities e.g. #sora that are amazing for early versions.

Simplifying.

To traverse from State A to State C you have to traverse through B.
And if you feed enough learning data into the model, you will get amazing results.
Modality is immaterial.

@david_chisnall

Even in Star Trek, the dangers of using natural language to command a computer are sometimes shown. That's how Moriarty was created.

@david_chisnall exactly.

And further language is deeply contextual and not just about the previous words - about the context of a conversation, about the social status and physical/visual and other sensory cues that inform how we interpret words as does what each party knows about the others

And even with all those other contextual cues people misinterpret each other all the time (was that person I met just now hinting that they like me. Is my partner serious just now or being playful)

@david_chisnall FANTASTIC writeup. You nailed the whole thing.

@david_chisnall

Humans are able to recover the meaning of an ambiguous text.

On the other hand , a new version of a language can make a program not be able to compile. The compiler cannot understand the meaning of the program.

Ambiguity thus leads to antifragility. We need to able to recover from a certain number of errors.

@david_chisnall

Summary in Meme format why voice controlled appliances are bad and you do not need to talk to Majel Barrett-Roddenberry to get tea

@david_chisnall Reminds me (imprecisely) of one of the Nancy buttons I've had for many years (paraphrased from memory):

“It's easy to write a program that matches the spec.
It's impossible to write a spec that says what you want.”

Natural language is the new "spec”: the person saying it thinks that what they are saying is unambiguous, and they are always, always wrong.

@david_chisnall great analysis

@david_chisnall Lofty ideals, but there's a reason Windows became more popular than Linux despite being worse. Ease. Same reason GUIs won out over command lines. It might be more efficient to do things the way you've described, but you're never going to get everyone to do it - they broadly want the easiest way, and there's probably no easier UI than natural language. Not sure LLMs, which are indeed horribly inefficient, are needed for natural language *input* though?

@david_chisnall Precisely. Using natural language to control things is cool until it goes wrong, and now you have a senior pilot sobbing and begging the airplane to stop trying to murder everyone. Mutatis mutandis for power generation, emergency services, healthcare, criminal justice, anything where the outcome actually matters. A tool that is fun but you would never use for anything important is a toy.

Using natural language to control ANYTHING is so complex, most of the time even human beings fail.... https://youtu.be/Hr4TG0zr-6c

or

https://youtu.be/okkIyWhN0iQ

/cc @david_chisnall

@david_chisnall
That's why nailing down requirements is often the harder part than writing the code.

Implementing it on a way they does not ruin the architecture and say it's rough characteristics can also be hard.

But again writing the code to move data around etc is generally not the issue.
@LordCaramac

@david_chisnall
That's why nailing down requirements is often the harder part than writing the code.

Implementing it on a way they does not ruin the architecture and say it's rough characteristics can also be hard.

But again writing the code to move data around etc is generally not the issue.
@LordCaramac

@david_chisnall

The underlying tech is *extremely* useful and could achieve things we can barely conceive of,

in the hands of capable programmers on modest hardware.

Instead, it's being used to free mediocre people from needing to interact with non-mediocre people.

edit: and all it's costing us is our ability to breathe! Even trade!

@david_chisnall Natural language is for talking with other people, who we shouldn't control. I think there's a related problem in there somewhere.

@david_chisnall You seem to presuppose that LLMs perform some form of interpretation, even if they might disambiguate the user's natural language incorrectly. I think this is still wildly optimistic. They just attempt to predict what another natural language user might reply, based on what is essentially a huge compressed Markov model. A human doing that would interpret the message first, but an LLM skips that step altogether.

@david_chisnall not all UI interactions need to be completely unambiguous. And even with natural language there are ways to confirm and be sure.

IMO this remains the only / best use case for LLMs. They’re decent at converting language to action and translating in that specific way. Let’s use them for that, with lots of controls, and basically nothing else.

@david_chisnall

"I'm sorry, I don't understand what you said, can you clarify?"

See how that works?

Yes, language is ambiguous, but when humans use it, it is also a two-way street, in which both parties are continuously tossing requests for information back and forth to remove the ambiguity and reach a common understanding.

@david_chisnall @spacelizard

https://youtu.be/QpWhugUmV5U

@david_chisnall if you're evil, a lot of those bugs look like features.

@david_chisnall@infosec.exchange I think something that laypeople assume about LLMs is that they are smart enough to figure out what they mean from ambiguous language. People that don't work with programming or other computer tasks assume it is easy for the computer to "just make it understand what I want it to do" when in reality it is almost impossible.

@david_chisnall 100% -- people want to believe that LLMs will magically "do the right thing" (or more surreal, "do what I think") despite all the ambiguity, possibilities, and complexity underlying the task.

@david_chisnall the other thing we think gets missed a lot is that language-based interfaces are the tyranny of the blank page, in the same way as command lines are

it can be anything (assuming it works, at least, which with LLMs is a big if)............ but first you have to know what you want it to be

@david_chisnall 🔥🔥🔥 indictment of LLMs 👆

"In C, compilers do this for undefined behaviour and it is widely regarded as a disaster. LLMs are built entirely out of undefined behaviour."

@david_chisnall

“Fruit flies like a banana.”

It is indeed unsolvable.

@david_chisnall
Spot on.