Here's everything I've figured out about the new OpenAI...

And a new release of LLM that adds support for those models - for anyone lucky enough to have a tier 5 OpenAI API account at least https://llm.datasette.io/en/stable/changelog.html#v0-16

12 September at 23:26 | Open on fedi.simonwillison.net

Jonathan Moore

@simon o1 gets my unbirthday question which none of OpenAI's other models have: "Do all people have the same number of unbirthdays?"

https://chatgpt.com/share/66e3652c-7ae0-8002-94dc-263f6c70422e

12 September at 23:46 | Open on mastodon.social

ijm

@simon the "Limit additional context in retrieval-augmented generation (RAG)" and "Hidden reasoning tokens" seem to be contradictory, or at least at odds: reduce the context, but you cannot use any reasoning tokens that came up while deciding what was and wasn't relevant.
I've been trying to find ways to improve rag index vectors, effectively trying to extract some internal state from a model as a sort-of pre-compiled version of the text - and these hidden reasoning tokens seem to be exactly related to what I'm looking for, but nope we're not allowed to see'm !

@simon the "Limit additional context in retrieval-augmented generation (RAG)" and "Hidden reasoning tokens" seem to be contradictory, or at least at odds: reduce the context, but you cannot use any reasoning tokens that came up while deciding what was and wasn't relevant.
I've been trying to find ways to improve rag index vectors, effectively trying to extract some internal state from a model as a sort-of pre-compiled version of the text - and these hidden reasoning tokens seem to be exactly related...

Expand text...

13 September at 15:34 | Open on mathstodon.xyz