 Trying something new: tomorrow (Friday 8th November) at 2pm Pacific we'll be hosting the first Datasette Public Office Hours - a livestream video session on Discord where @alexgarciaxyz and myself will live code on some Datasette projects, demonstrate some new features and generally hang out with people to chat about the project - event details here: https://discord.gg/udUyEnv3?event=1304134449453072435 Legitimately impressed by this result from Claude - I prompted "Prototype an SVG social media card to represent a Datasette table" and it generated this artifact, using its existing baked-in knowledge of what Datasette is The problem with most AI image tools is that you get very loose control over what they put out. Generating SVG helps address that because you can "take over" and make any edits you want to at the vector or text level  @simon I wish open graph tags were in a better state to render SVG files. (maybe they are now and I need to re-test it.) I played with this some time ago, thinking I could skip the HTML template to PNG dance to get nice open graph images. It's reassuring how many candidates for local office around here have (competent) websites for their campaigns People running for positions that pull in just a few thousand votes total (Half Moon Bay City Council District 3 appears to cover ~40 city blocks) still see value in building a website rather than just having a Facebook page or similar - this one for example https://www.paulforhmb.com Feels like a small signal of a healthy web  I wrote this if for no other reason than to stop reserving space in my head for some of these links. Now I'll just link to my blog instead (and hopefully push for them to be visible enough that I no longer need to deep link to these pages). https://www.jonafato.com/2024/11/06/psf-and-dsf-membership-application-links Finding these links is often the most time consuming part of this process. Now that you have them, it should only take a minute or two to fill out the form. Please do so, and please share them with your friends. Here's a neat trick for libraries that are under-documented but have comprehensive tests: use the CLI combo of llm and files-to-prompt to generate your own private documentation directly from their test suite! https://til.simonwillison.net/llms/docs-from-tests Full transcript here: https://gist.github.com/simonw/351cffbd254af5cbf329377fb95fcc13 OpenAI reduced a slightly confusing feature today called "Predicted Outputs", which lets you spend more money on a prompt to have it return faster if the output is mostly predictable based on the input (e.g. a prompt that makes edits to some code) - detailed notes here: https://simonwillison.net/2024/Nov/4/predicted-outputs/ Claude 3.5 Haiku is out, with a couple of surprises: 1. It's priced differently from Claude 3 Haiku. 3.5 Sonnet had the same price as 3 Sonnet, but 3.5 Haiku costs ~4x more than 3 Haiku did 3.5 Haiku supposedly beats 3 Opus though, and Opus cost 15x the new Haiku price! I released a new version of llm-claude-3 adding support for the new model (and fixing an attachments bug): https://github.com/simonw/llm-claude-3/releases/tag/0.8 llm install --upgrade llm-claude-3 I also added 3.5 Haiku to my LLM pricing calculator: https://tools.simonwillison.net/llm-prices @simon The price adjustment was disappointing. If the price remained the same, it would have been a game changer in cases like pair programming (aider) etc. Because I can never figure out what's going to happen with Daylight Savings Time changes I got Claude to build me my perfect California DST app https://tools.simonwillison.net/california-clock-change Claude transcript here https://gist.github.com/simonw/9510723176f5b44ac1ebc495c95a4bc7  Added a section helping explain what's going on with our dog Cleo: https://gist.github.com/simonw/d680d5f27b50de4efc26ed2a673292fd Published some notes on Docling, a rather nice MIT licensed Python PDF document / table extraction library from IBM https://simonwillison.net/2024/Nov/3/docling/ @simon How does the Markdown output from Docling compare with the HTML that you've gotten out of Gemini for PDF documents? Does Docling do a good job of recognizing headings, lists, etc.?  If you need an idea or nudge, feel free to reach out. Many of you write about cool things here, but they never make it to an article, even though you are still doing 99% of the work. Me: "This is great, please blog about it, so I can share it more easily." @webology My problem is that despite having many things I could write about, I'm too burned out by the state of things to put any words down. Sigh.
Show previous comments
 @webology Thanks for the nudge. For a subsequent trick, I'll make that site a little less awful to look at.  @webology Great timing, I just started a microblog on my site so I would feel more comfortable writing shorter things than those found on my main blog. I did mine in a subdir instead of a subdomain. https://www.bobmonsour.com/microblog/  @webology "Write and publish before you write your own static site generator or perfect blogging platform" Guilty (for the past 10 years) Anthropic added PDF support to their Claude API this morning, I have a new release of the llm-claude-3 plugin that supports that as a new attachment type: llm install llm-claude-3 --upgrade Details: https://simonwillison.net/2024/Nov/1/claude-api-pdf-support-beta/  Google Gemini (on Android) now integrates with Google Home https://simonwillison.net/2024/Nov/1/smart-home-prompt-injection/ They've excluded security devices (cameras, locks) but it can operate all sorts of other "smart devices" - who's going to be first to demonstrate a prompt injection attack against a coffee maker? I first talked about the security risks posed by LLM-assistants in this piece in April 2023 https://simonwillison.net/2023/Apr/14/worst-that-can-happen/ I’m working on my first machine learning feature using Keras on a team with no ML or data science expertise, and Datasette by @simon has been an amazing tool to help us work through visual validation and data curation tasks. @shiftingedges that's awesome! I had a hunch it would be good for that kind of work, but I haven't seen it used like that myself yet I feel the need to formally apologize to Halloween Wars, the reality TV competition show where teams consisting of a pumpkin carver, a sugar artist and a cake decorator compete to create ludicrous vignettes of Halloween scenes by combining their individual specialities I have been bad mouthing them for dropping the pumpkin carver role I just found out they only did that for one year in 2021 because Covid meant they had to film outside of pumpkin season Hard-hitting coverage of the Halloween Wars format changes here from the Miami Student newspaper https://www.miamistudent.net/article/2022/10/halloween-wars-review Got a bit behind on my weeknotes, so here are my monthnotes for October instead https://simonwillison.net/2024/Oct/30/monthnotes/ I added multi-modal (image, audio, video) support to my LLM command-line tool and Python library, so now you can use it to run all sorts of content through LLMs such as GPT-4o, Claude and Google Gemini Cost to transcribe 7 minutes of audio with Gemini 1.5 Flash 8B? 1/10th of a cent.
Show previous comments
If you are still LLM-skeptical but haven't spent much time thinking about or experimenting with these multi-modal variants I'd encourage you to take a look at them Being able to extract information from images, audio and video is a truly amazing capability, and something which was previously prohibitively difficult - see XKCD 1425 https://xkcd.com/1425/ @simon Something I've been meaning to ask.. is there a decent guide to which models are best suited for which tasks? As in, "gemini models are better for extracting content from video/audio, etc", including model versions, sizes, etc. @simon Does video work? I tried both Gemini pro and flash, but I only got some error message. Do I need a paid account to use video scraping? (Image works as expected.) I built a little browser-based tool for playing with the audio output from the OpenAI GPT-4o audio preview model - you can set a system prompt and a regular prompt, play the resulting audio, download the wav file and also export out the underlying JSON Tool is here (you'll need to provide your own OpenAI API key, stored in localStorage): https://tools.simonwillison.net/openai-audio-output Notes on how I built it (with Claude) here: https://simonwillison.net/2024/Oct/28/prompt-gpt-4o-audio/ Bonus tool: if you save the raw API JSON as a Gist you can add the Gist ID to this URL to serve up a page that lets other people play back your audio Here's my example with a system prompt specifying a "thick French accent": https://tools.simonwillison.net/gpt-4o-audio-player?gist=4a982d3fe7ba8cb4c01e89c69a4a5335 Wrote a tiny new LLM plugin this morning: llm-whisper-api, which lets you do this (if you have an OpenAI API key configured already): llm install llm-whisper-api @simon In principle, human transcription as a job or service is also dead. I've always wished for something like this to turn podcasts into texts. Had played around with opensource like cmusphinx a long time ago. But the quality wasn't particularly good then and it was also very slow. It's impressive how it has developed. I built a new plugin for LLM called llm-jq, which lets you pipe JSON into the tool and provide a short description of what you want, then it uses an LLM to generate a jq program and executes that against the JSON for you https://simonwillison.net/2024/Oct/27/llm-jq/ Example usage: llm install llm-jq
Show previous comments
@simon do you pass all or parts of the json to the llm? Or json schema with the instructions? How does it work with large json data? Use a sample? |
Gregory's Server