@simon it was also shared on /r/claudeai where a substantial...

@simon it was also shared on /r/claudeai where a substantial portion of the community are convinced sonnet 3.5 has degraded significantly in recent weeks. https://old.reddit.com/r/ClaudeAI/comments/1f1shun/new_section_on_our_docs_for_system_prompt_changes/

Per the rep "We've also heard feedback that some users are finding Claude's responses are less helpful than usual. Our initial investigation does not show any widespread issues. We'd also like to confirm that we've made no changes to the 3.5 Sonnet model or inference pipeline."

Like 26 August at 20:41 | Wall-to-wall | Open on fosstodon.org

2 comments

Simon Willison

@asb yeah the superstition about the models getting weaker is always so interesting to me - especially when the LLM reps say things like "we haven't deployed any changes, it's the same exact binary on disk" - here's another of those from April https://simonwillison.net/2024/Apr/15/jason-d-clinton/

26 August at 20:44 | Open on fedi.simonwillison.net

Clifford Adams

@simon @asb
My guess is that the "reduction in quality" is really people becoming more critical of the outputs. In the beginning, LLMs seem magical, and one is willing to overlook many flaws. Over time one becomes less astonished by the good outputs (yes, it can create a sonnet about a duck who is a pirate), and more critical of the flaws (the output doesn't fully match the formal definition of a sonnet).

27 August at 0:31 | Open on fosstodon.org