 I added multi-modal (image, audio, video) support to my LLM command-line tool and Python library, so now you can use it to run all sorts of content through LLMs such as GPT-4o, Claude and Google Gemini Cost to transcribe 7 minutes of audio with Gemini 1.5 Flash 8B? 1/10th of a cent. 20 comments
@ame I want to do a bit more with token accounting - maybe store them in separate database columns - but I'm not so keen on doing the price calculations in the tool because I can't promise correct results, since the prices often change sometimes without any warning I'd be OK outsourcing that to a plugin though @simon first of all thank you very much. In your llm logs output you have "total_tokens". I do not have it, I have a lot of time "totalTokenCount": 3522" https://gist.github.com/aborruso/6004d5daacc1a7096a07b2af381dec2b Am I doing something wrong @aborruso that output is different for different models - I have a future plan to normalize those and store them separately in the database @simon Sorry, another stupid question: and in my case I have to add up all the totalTokenCounts? Thank you If you are still LLM-skeptical but haven't spent much time thinking about or experimenting with these multi-modal variants I'd encourage you to take a look at them Being able to extract information from images, audio and video is a truly amazing capability, and something which was previously prohibitively difficult - see XKCD 1425 https://xkcd.com/1425/  The LLM Python library supports attachments now as well https://llm.datasette.io/en/stable/python-api.html#attachments  @prem_k more docs here: https://llm.datasette.io/en/stable/plugins/advanced-model-plugins.html#attachments-for-multi-modal-models Implementations are spread out across different plugins, eg https://github.com/simonw/llm/blob/a44ba49c21f8d4ac30c8e41bfa5599c258ce53cc/llm/default_plugins/openai_models.py#L338 and https://github.com/simonw/llm-gemini/blob/ce82727a6950c7769a8e40bf030591d0e6f83e5e/llm_gemini.py#L135  @djh those are all very valid reasons to be skeptical! The only reason I'll consistently push back at is the idea that these things aren't useful at all @simon Something I've been meaning to ask.. is there a decent guide to which models are best suited for which tasks? As in, "gemini models are better for extracting content from video/audio, etc", including model versions, sizes, etc.  @simon Does video work? I tried both Gemini pro and flash, but I only got some error message. Do I need a paid account to use video scraping? (Image works as expected.) @xsc video should work, what file format were you trying? Currently needs to be less than 20MB - that's a temporary limitation of my llm-gemini plugin @simon I was using an MP4 of 5 mb size. The error just says "internal error" I downloaded the video from here https://www.pexels.com/video/catching-and-releasing-a-big-carp-fish-in-the-lake-5538137/ @xsc I've seen a few of those "Internal error" messages too - I think it's Gemini being a little bit flaky, sometimes resubmitting works fine the second time @simon I was using the following command > llm 'please explain what is happening in the video' -a man-in-water.mp4 -m gemini-1.5-flash-latest Does it look like it should work? @xsc yes, if you have the llm-gemini plugin installed and configured with an API key You could try using this script here (or using Google's AI Studio tool) ti check it's not an LLM bug: https://til.simonwillison.net/llms/prompt-gemini |
Gregory's Server
@simon Since you have the tokens readily available what do you think about including the pricing calculator directly inside `llm`?
aider is doing a similar thing by directly showing you how much you were billed for each response.