The Llama models I tried both did terribly, but Gemini...

The Llama models I tried both did terribly, but Gemini 1.5 Flash 8B wins for weird charm (even if it doesn't really look like a pelican at all)

Cerebras Llama 3.1 70B and Llama 3.1 8B:

These both chose blue backgrounds. They are weird misshapen blobs, thoough the 70b one at least has elements of a pelican in there somewhere.

And a special mention for Gemini 1.5 Flash 8B:

This is a weird yellow heart-shaped creature with a deformed face which is somehow still quite charming

Like 26 October at 0:02 | Open on fedi.simonwillison.net

2 comments

Drew Breunig

@simon This is great! For awhile I was testing them by asking them to draw a deserted island in Processing. It was hilarious.

26 October at 0:07 | Open on note.computer

Simon Willison

Paul Calcraft extended this idea into an implementation of Pictionary where different vision LLMs generate SVGs and race to guess what the others are drawing and it is absolutely brilliant https://twitter.com/paul_cal/status/1850262678712856764

Screenshot of LLM Pictionary game showing Round 1 with a simple drawing of an orange giraffe-like figure with brown spots against blue background. Multiple AI models (Claude 3.5 Sonnet, GPT-4o, Gemini Flash/Pro, Llama) attempt to identify with responses including "duck", "swan", "rocket", "snake", "snail", and "giraffe". GPT-4o guessed correctly first. Website URL x.com/paul_cal shown

Screenshot of LLM Pictionary game showing Round 5 with a simple drawing of an ocean scene with yellow sun, bird, and waves. Multiple AI models (Claude 3.5 Sonnet, GPT-4o, Gemini Flash/Pro, Llama) attempt to identify the image with responses like "Ocean", "Sky", "Sun", and "Beach". Claude 3.5 Sonnet (June 24) guessed correctly first.

26 October at 21:15 | Open on fedi.simonwillison.net

Go Up