 Today we got what must be the most alarming first line in a newly file sec issue to #curl: "To replicate the issue, I have searched in the Bard about this vulnerability" ... followed by a complete AI hallucination where Bard has dreamed up a new issue by combining snippets from several past flaws. Creative, but hardly productive. Closed as bogus. 45 comments
 @timhaines @bagder itโs happening, Iโve got one reported on a paid bug bounty programme where they asked Bing AI (lol) to find a vulnerability and then just copypasta the outputs. Unsurprisingly it just made up the issue. @GossiTheDog @timhaines it is fine as long as that they mention "AI", "Bard" or similar early on in the report so that we can discard it quicker. =) @bagder @GossiTheDog @timhaines I would tell them next time to ask the Bard to write about the issue talking like a pirate, so that at least you can laugh before blocking them to death.  @GossiTheDog @timhaines @bagder There should almost be a fine for people knowingly submitting false or AI generated reports. @dascandy42 @timhaines @bagder put it this way, a lawyer submitted court documents using ChatGPT and when called out about it - because the citations it used was made up - said they didnโt know AI could be wrong. Bard told me I died of cancer in 2021, so hello from a ghost. ๐ป @dascandy42 @GossiTheDog @timhaines fortunately, at least on hackerone, there's a "reputation" for the hacker that gets a dent when they do this. @bagder @GossiTheDog @timhaines I typed "almost", because this is going to create a situation similar to StackOverflow, where new users are almost unable to report anything. @dascandy42 @GossiTheDog @timhaines yeah, that would not be good either! โน๏ธ @GossiTheDog @timhaines @bagder Can you charge these people instead, if the claim turns out to be bogus? After all, they wasted your time and did not do any research on their own.   @JungleGeorge24 I considered it wiser to not dig too deep into the details behind this...  @bagder I hope this isn't a sign of things to come... We'll be wasting a lot of time. @jan I'm pretty sure this will get worse before it can get better - and I bet in future reports they will hide the fact it came straight from AI better...   @bagder ugh, wading through nonsense like this with maybe something real hiding in the new reports  There has to be clear unavoidable disclaimers of the form, "Do not trust anything Bard/Bing says. Bard/Bing will often make up fictional answers, nothing it says is to be trusted". But I guess that wouldn't be great PR. Especially if you're in the unfortunate position of having named your chat bot the same as your search engine. @bagder "README.md typo MR" makers after meeting "ChatGPT-powered MR" makers: @bagder when we recruit we send the interviewee a bunch of code puzzles (no pressure to complete them but works in their favour if they do some). Guess how long it was before we started seeing chatgpt answers? @bagder and yes, we do now embed hidden text on the code instructing the llm to refer to things in a way that's obvious. And no, no one has spotted that yet   @bagder I see this as a net positive. as this hallucinated crap is posted and then crawled over and consumed by google it will soon become impossible to google real vulnerabilities and PoCs due to zero signal-to-noise ratio. ๐  @bagder No, there's a flaw there. It's just not in curl, it's in users. It's a whole new level of threat, one that comes from an attacker that can social engineer users into doing harm without itself being malicious or even sentient. Can we CVE users? @snail it was submitted as a security problem over at hackerone and we have not disclosed it, simply because its not worth spending time on   |
Gregory's Server
@bagder beg bounty 2023