Email or username:

Password:

Forgot your password?
daniel:// stenberg://

We disclosed this #hackerone report against #curl when someone asked Bard to find a vulnerability, and it hallucinated together something:

hackerone.com/reports/2199174

18 comments
Kevin P. Fleming

@bagder And the report is that the fixes for the vulnerability are posted on the Internet? This is so ridiculous.

daniel:// stenberg://

@kevin it was filed *before* the actual 38545 CVE had been made public though

Dylan Van Assche

@bagder This is why you can never trust a #LLM... They dump so much #inaccurate #information or even #wrong information :(

Kornel

@bagder {"error": "too many requests"}

You've hacked hackerone (remote DoS, 9.8 CVSS)

Mateus Melchiades :gopher:

@kornel @bagder you mess with the curl, you get the curl

Hayley Question-Mark

@bagder At least they redacted your name, I guess..?

left-wing math nerd

@bagder “I have searched in the Bard about this vulnerability”. Right there is the problem. LLMs are not search engines. This is similar to the attorney that “searched” for case law using ChatGPT and ended up filing a legal argument full of references to made up cases.

nepi

@bagder@mastodon.social thanks for posting this, i needed a chuckle with my morning coffee!!

edit: the chuckle has worn off and now it's "oh we're doomed aren't we"

derekheld

@bagder it’s all the weirder because they aren’t even trying to report a new vulnerability. Their complaint seems to be that detailed information about a “vulnerability” is public. But that’s how public disclosure works? And open source? Like are they going to start submitting blog posts of vulnerability analysis and ask curl maintainers to somehow get the posts taken down???

daniel:// stenberg://

@derekheld they reported this before that vulnerability was made public though

derekheld

@bagder oh as in saying the embargo was broken but with LLM hallucinations as the evidence?

Brodie Robertson

@bagder I have 0 doubts that this will become a more and more common occurence

kurtseifried (he/him)

@bagder I remember when I was at Red Hat I did a thought experiment: what is the minimal amount of work an attacker could do to cause the maximum amount of effort by a security team?

This was over 10 years ago and a lot of what we were experiencing and what I came up with back then is now trivial for attackers thanks things to these LLM‘s.

And the problem is you can’t have a skill testing question or something because occasionally somebody will find a gem in the rough and report it, and risk of missing that is seen as not acceptable by most people.

I don’t know what the future bolts for open source security reporting, but I have a suspicion. Things are gonna have to change in the next few years. People are going to get burnt out.

Edit: for readability

@bagder I remember when I was at Red Hat I did a thought experiment: what is the minimal amount of work an attacker could do to cause the maximum amount of effort by a security team?

This was over 10 years ago and a lot of what we were experiencing and what I came up with back then is now trivial for attackers thanks things to these LLM‘s.

Adam Piggott

@bagder Bard doing bard things - writing entertaining stories that are nothing more than myth. All it needs is a lute and a penchant for rough taverns.

Patrick $8 :verified:

@bagder I suspect the reporter's last comment in that thread was also written by an LLM

Haelwenn /элвэн/ :triskell:
@bagder I could understand using some kind of AI to get something similar to a fuzzer but this is utterly ridiculous…
Ingvar

@bagder On the plus side, they pretty much started with "I asked the Bard". Imagine if that bit had not been there?

Go Up