Email or username:

Password:

Forgot your password?
Simon's posts Post Back to profile
Simon Willison

Anthropic released the system prompt for their various consumer LLM chatbot apps today, and they're a really fun read. Made some notes on them here: simonwillison.net/2024/Aug/26/

Here's how Claude 3.5 Sonnet deals with controversial subjects:
26 August at 20:24 | Open on fedi.simonwillison.net
9 comments
Simon Willison

And here's a fun little hint at some of the annoying behaviour in the base model that they've tried to knock out of it with some system prompt instructions

Seriously, stop saying "certainly"!
26 August at 20:24 | Open on fedi.simonwillison.net
Jon Udell

@simon Yeah, no kidding. Any sign of a customizable base user prompt a la ChatGPT?

And, follow-up question, assuming you're using that feature of ChatGPT, how helpful do you think it has been?

26 August at 20:32 | Open on social.coop
Simon Willison

@judell you can get a bit of that with Claude "Projects" - I have one which thinks it's a manatee who's an expert at vegetarian recipes and salads

I don't use it for ChatGPT because I don't like the idea that my interactions with the models might be biased in unexpected ways by my custom instructions - I want to gain as much experience as possible with the base model that everyone else is using

26 August at 20:37 | Open on fedi.simonwillison.net
Quinn Comendant

@simon I haven’t had much success using negative prompting. I’m surprised they get this to work.

27 August at 0:17 | Open on mastodon.social
Simon Willison

@com yeah same here, telling models not to do things isn’t that effective in my experience

I’m pretty sure I’ve still seen Claude say “certainly”!

27 August at 0:19 | Open on fedi.simonwillison.net
Alex Bradbury

@simon it was also shared on /r/claudeai where a substantial portion of the community are convinced sonnet 3.5 has degraded significantly in recent weeks. old.reddit.com/r/ClaudeAI/comm

Per the rep "We've also heard feedback that some users are finding Claude's responses are less helpful than usual. Our initial investigation does not show any widespread issues. We'd also like to confirm that we've made no changes to the 3.5 Sonnet model or inference pipeline."

@simon it was also shared on /r/claudeai where a substantial portion of the community are convinced sonnet 3.5 has degraded significantly in recent weeks. old.reddit.com/r/ClaudeAI/comm

Per the rep "We've also heard feedback that some users are finding Claude's responses are less helpful than usual. Our initial investigation does not show any widespread issues. We'd also like to confirm that we've made no changes to the 3.5 Sonnet...

26 August at 20:41 | Open on fosstodon.org
Simon Willison

@asb yeah the superstition about the models getting weaker is always so interesting to me - especially when the LLM reps say things like "we haven't deployed any changes, it's the same exact binary on disk" - here's another of those from April simonwillison.net/2024/Apr/15/

26 August at 20:44 | Open on fedi.simonwillison.net
Clifford Adams

@simon @asb
My guess is that the "reduction in quality" is really people becoming more critical of the outputs. In the beginning, LLMs seem magical, and one is willing to overlook many flaws. Over time one becomes less astonished by the good outputs (yes, it can create a sonnet about a duck who is a pirate), and more critical of the flaws (the output doesn't fully match the formal definition of a sonnet).

27 August at 0:31 | Open on fosstodon.org
Go Up