Email or username:

Password:

Forgot your password?
Mozilla's posts Post Back to profile
Mozilla

Firefox 130 is bringing a game-changing feature: automatic alt-text generation for images using a fully private on-device AI model! 🙌🏾

Initially available in the built-in PDF editor, our aim is to extend this to general browsing for screen reader users. hacks.mozilla.org/2024/05/expe

3 June at 9:35 | Open on mozilla.social
177 comments
Andreas M. Heitmann :batman:
AlexTECPlayz

@mozilla This is how it's done! Private, open-source AI models running locally.

Q: How much storage do the models take? (EDIT: 200MB according to the post - yeah, in this case, this better be a downloadable 'module' instead of being built-in) Could you make this feature optional, which would require the user to opt-in and download or delete the model(s) themselves? I don't want Firefox to go the Microsoft Edge route, where they shovel every feature under the sun, the user has no choice, and there is no way to reduce the storage occupied by the browser.

@mozilla This is how it's done! Private, open-source AI models running locally.

Q: How much storage do the models take? (EDIT: 200MB according to the post - yeah, in this case, this better be a downloadable 'module' instead of being built-in) Could you make this feature optional, which would require the user to opt-in and download or delete the model(s) themselves? I don't want Firefox to go the Microsoft Edge route, where they shovel every feature under the sun, the user has no choice, and there...

3 June at 9:43 | Open on techhub.social
v

@alextecplayz @mozilla on that last point:
> As the project grows, we anticipate the browser will store more models, which can take up significant space on disk. We plan to add an interface in Firefox to manage downloaded models so our users can list them and remove some if needed.
(from the post)

3 June at 9:53 | Open on merveilles.town
AlexTECPlayz

@v @mozilla I skimmed through the article so I may have missed it, but that is pretty good! I'm hoping that they won't include any model by default purely to save space, unless they can make it small enough (smaller than 50-75MB), and on the first startup, to prompt the user when setting Firefox up, if they want to 'enable the feature', which would download the AI model(s).

3 June at 9:56 | Open on techhub.social
v

@alextecplayz @mozilla the flow for translations they give is this:
> For example, if you run Firefox in French and visit an English page, Firefox will ask if you want to translate it to French and download the English-to-French model (~20MiB) alongside the inference runtime. This is a one-shot download: translations will happen completely offline once those files are on disk.
which seems very reasonable!

3 June at 9:58 | Open on merveilles.town
Matthias

@mozilla Looks interesting at first glance.

Thanks for being this open and transparent about the process, used model etc.

3 June at 9:44 | Open on mastodon.world
Orca🌻 | 🏴🏳️‍⚧️

@mozilla@mozilla.social Well I hope that does work fine.
I don't quite like AI features, but users with visual impairment* does need more love from software projects, especially browsers like Firefox.

* And other impairments, of course!

3 June at 9:44 | Open on nya.one
Mémoriser : Gauche Droite

@mozilla

Qu'est-ce que vous ne comprenez pas, lorsque l'on vous dit que l'on ne veut pas d'IA ?

3 June at 9:47 | Open on gayfr.social
Thomas Hareau

@MGD @mozilla chai pas ça a l'air pas mal non ? Pour moi ça change pas la vie, mais pour quelqu'un de malvoyant ça doit être super utile ?

3 June at 14:39 | Open on piaille.fr
Sparky 💡

@mozilla Finally a good use of Machine Learning.
I hope it works well for those that need it.

Of course, it would be ideal if everyone wrote their own Alt Text and all platforms allowed users to do so, but this is a great solution for cases where people don't want to, have the spoons to or can't do it themself.

Assuming it works of course, but I am hopeful.

Edit: I posted second thoughts about it seperately. tl;dr: Might be useful if it admits failure instead of hallucinating.

@mozilla Finally a good use of Machine Learning.
I hope it works well for those that need it.

Of course, it would be ideal if everyone wrote their own Alt Text and all platforms allowed users to do so, but this is a great solution for cases where people don't want to, have the spoons to or can't do it themself.

3 June at 9:47 | Open on kind.social
Gloopsies :fedora:

@Bright5park @mozilla

Afaik there is no option for alt text for embedded images in pdf files so this is a game changer for that use case, but I agree that manual alt text is always a better option when available

3 June at 14:43 | Open on fosstodon.org
Tom Ritchford

@mozilla Nice work!!!

This would be particularly useful for postings to Mastodon, where alt-text is much socially desirable.

3 June at 9:49 | Open on toot.community
Julian

@TomSwirly @mozilla but I think this shouldn't be used as an excuse for writing alt texts when you post pictures on Mastodon... a neural network will never be 100% correct at guessing what a picture shows and maybe also what you want to tell the person looking at it...

3 June at 10:49 | Open on social.jugendhacker.de
Tom Ritchford

@j_r @mozilla I always write alt-text when I post images on Mastodon!

However, I share things that don't have alt-text if I think they're important enough.

3 June at 10:55 | Open on toot.community
ferricoxide

@mozilla@mozilla.social

Ah, so, reader-side rather than something that, say, a Fedi user c/would use when posting images?

3 June at 9:49 | Open on evil.social
peacememories

@mozilla you do realize that incorrect alt-text can be worse than no alt-text? Sometimes critically so? I do hope this will be strictly opt-in and alt-texts generated by the model will be clearly marked, if this ever goes into stable Firefox

3 June at 9:55 | Open on chaos.social
Baron Henrik von Boyage

@peacememories @mozilla

I guess one would still have the possibility to review the text before setting it.

3 June at 10:20 | Open on eliitin-some.fi
peacememories

@henrik @mozilla Oh in an interactive environment, sure, like the example of adding images to a PDF. But if this tries to automatically add alt-texts to websites there is by definition no review that can be done.

3 June at 10:22 | Open on chaos.social
Carla, for now

@henrik People don't even review the OCR-generated alt text on Mastodon.

3 June at 13:30 | Open on tech.lgbt
Baron Henrik von Boyage

@i_cannot_today

I do, it's just a piece of crap, so it never gets put anywhere. 😅

3 June at 13:35 | Open on eliitin-some.fi
Nickname

@peacememories
From reading the first section of the article, I believe that it is meant for people using screen readers, if you use a screen reader I doubt that you take the pronunciation at face value as well. And now instead of the Screen reader reading "bla bla IMAGE bla bla" it will offer an alt text.
But yeah obviously a way to differentiate between existing and locally generated ALT texts would be good, with the option to prompt for a new local alt text etc.
@mozilla

3 June at 10:24 | Open on mastodon.bayern
Anreji :blobcatheart:

@peacememories @mozilla iOS and Android have been using similar models for quite some time and the descriptions have been proven useful. Firefox isn't reinventing the wheel here, they are matching what the industry has already been doing for years. Blind and visually impaired people rely on AI generated alt text all the time already. Of course human provided alt text is preferred, but in the grand scheme of the internet it is still really rare, despite decades of education regarding the topic :/

3 June at 13:01 | Open on tech.lgbt
d@nny "disc@" mc²

@mozilla alt text cannot be automatically generated without human input because the function of alt text is highly contextual. if you actually gave a shit about a freer more independent web you'd support projects like @hannah's distributed alt text database which is currently supported via browser extension. it looks like you're too late now to take it over but if you have any funding left for non-AI bullshit i'm sure the 501(c)(3) would absolutely love your support as well as built-in browser integration social.alt-text.org/@hannah/11

@mozilla alt text cannot be automatically generated without human input because the function of alt text is highly contextual. if you actually gave a shit about a freer more independent web you'd support projects like @hannah's distributed alt text database which is currently supported via browser extension. it looks like you're too late now to take it over but if you have any funding left for non-AI bullshit i'm sure the 501(c)(3) would absolutely love your support as well as built-in browser integration

3 June at 9:56 | Open on circumstances.run
d@nny "disc@" mc²
s92

@hipsterelectron @mozilla @hannah I like how you're yelling at one of the few organizations that go to great lengths to build a completely independent browser that they "don't give a shit" about the free web.

If they don't give a shit then literally nobody on earth does.

3 June at 10:51 | Open on c.im
d@nny "disc@" mc²

@sasha92 @mozilla i yelled at them on twitter a few days ago too i'm trying to get them to change so we don't have to throw the baby out with the bathwater x.com/hipsterelectron/status/1

3 June at 10:59 | Open on circumstances.run
d@nny "disc@" mc²

@sasha92 @mozilla and uhhh clearly hannah gives a shit which is why she built the distributed alt text database. mozilla clearly isn't doing its job as an org so individual hackers on the internet are doing it for them and that's obviously much harder. i spelled out how they can do better, you're giving them a pass because they did something cool once. guess which one of us cares more about mozilla's success?

3 June at 11:01 | Open on circumstances.run
Emi

@hipsterelectron @sasha92 @mozilla There is billions of images online and an external human can't describe each picture that's missing alt text.
That project has a different goal than Mozilla's alt-text AI and I am sure you can use both - human descriptions with that project for the few images that will have it and Mozilla's AI for the rest.

3 June at 11:29 | Open on tilde.zone
d@nny "disc@" mc²

@paper @sasha92 @mozilla you do know there are billions of humans right? wikipedia is widely used to train statistical models but it too is the compilation of many humans over time. adding alt text to an image takes about as much effort as editing a page (and often much less). do the math

3 June at 11:34 | Open on circumstances.run
d@nny "disc@" mc²

@paper @sasha92 @mozilla also, as noted above, it is generally accepted by experts on alt text (not experts on "AI", who have a conflict of interest) that it cannot be machine generated in any remotely meaningful way. this is like providing an escalator to wheelchair users

3 June at 11:37 | Open on circumstances.run
Emi

@hipsterelectron @sasha92 @mozilla There are 3.2B images uploaded in a day (1.2T /year), many of them are repeating, google has 130B indexed. You can't describe all of that. Sure, human description will probably be better in many cases, but AI descriptions are still very useful.

Also, I doubt that project will get as many people editing it as Wikipedia has, so it can be great for a few popular images, memes, etc. but it can never cover random images on social media and websites without alt text.

3 June at 11:54 | Open on tilde.zone
d@nny "disc@" mc²

@paper @sasha92 @mozilla if you're not blind or visually impaired you absolutely don't get to make that call (and single individuals do not represent an entire community). hannah's project was built in direct conversation with the disabled people it serves. you're finding reasons to trash it to justify your preconceived notions and i think that's a really terrible thing to do. blocking now

3 June at 12:01 | Open on circumstances.run
drevil

@hipsterelectron @paper @sasha92 @mozilla nobody is trashing on anything bud, they are just saying that it would be physically impossible for any group of people, regardless of how large they are, to add a description to a fraction of the imagines uploaded every day.

You could, instead, argue that it would be nice to use image detection to match images with multiple databases that contain descriptions, so if you don't feel like using the model (and/or you don't like what it gives you) you can try to check a community provided one, if any exist.

@hipsterelectron @paper @sasha92 @mozilla nobody is trashing on anything bud, they are just saying that it would be physically impossible for any group of people, regardless of how large they are, to add a description to a fraction of the imagines uploaded every day.

You could, instead, argue that it would be nice to use image detection to match images with multiple databases that contain descriptions, so if you don't feel like using the model (and/or you don't like what it gives you) you can try...

3 June at 12:39 | Open on tilde.zone
Emi replied to drevil

@chickfilla @hipsterelectron @sasha92 @mozilla I can't reply to them as they blocked me, but I just wanted to say that I do see the value in that project and I like what they are doing. I am just saying that while this project will not give as accurate alt texts as that human made database, I think it is still valuable because of how many images without alt text exist out there.

3 June at 14:06 | Open on tilde.zone
drevil replied to Emi

@paper @hipsterelectron @sasha92 @mozilla Oh and I agree with you entirely, I wrote another reply where I basically said the same thing. I think it is a very noble and useful cause, my issue was with the nature of their replies instead.

3 June at 14:25 | Open on tilde.zone
d@nny "disc@" mc² replied to drevil

@chickfilla @paper@tilde.zone @sasha92@c.im @mozilla please read up on hannah's work before suggesting things like you're the first person to ever consider this problem. "image detection" like you describe is totally separate from "AI" bullshit (so why are you even mentioning it like it's some sort of gotcha?) and is very much a component of hannah's existing implementation

3 June at 14:15 | Open on circumstances.run
drevil replied to d@nny "disc@" mc²

@hipsterelectron @mozilla I think you misunderstood what I am trying to say. I am suggesting that one could use it to relate an image with community supplied alt texts should you want that as an option and they be available.

Also I am not looking for any gotchas, I do think a project focused on providing image descriptions is a noble cause, my problem comes with your idea that a model has no use for this. If this creates an image description that the poster likes and feels like it represents the image they are about to post well, then what's the harm?

You are acting extremely defensive when nobody, and I mean nobody, is discrediting nor even criticizing their project, just pointing out obvious limitations that do not in any way discredit its intentions.

@hipsterelectron @mozilla I think you misunderstood what I am trying to say. I am suggesting that one could use it to relate an image with community supplied alt texts should you want that as an option and they be available.

Also I am not looking for any gotchas, I do think a project focused on providing image descriptions is a noble cause, my problem comes with your idea that a model has no use for this. If this creates an image description that the poster likes and feels like it represents the...

3 June at 14:24 | Open on tilde.zone
d@nny "disc@" mc² replied to drevil

@chickfilla ok then i have no clue what you're trying to say. are you just trying to paint me as defensive? sorry for assuming you were acting in good faith

3 June at 14:26 | Open on circumstances.run
s92

@hipsterelectron @paper @mozilla What is "an expert on alt text"? Is that a linguist?

3 June at 12:04 | Open on c.im
Yusuf Toropov

@hipsterelectron @paper @sasha92 @mozilla "generally accepted by experts on alt text" that this can't actually work sounds a bit weasel-languagey, sorry. I'm not an expert on anything and I'm going to use this. That's if it's okay with @hipsterelectron , I mean.

Not every tool incorporating AI is evil. Some of it is both useful and transparent. AT THE SAME TIME. Mind boggling.

3 June at 14:48 | Open on toot.community
s92

@hipsterelectron @mozilla I didn't comment on the merits of the project you propose they adopt. Saying "do exactly as I say" or you don't give a shit is not the most productive approach, to say the least.

3 June at 12:04 | Open on c.im
d@nny "disc@" mc²

@sasha92 @mozilla it's called constructive criticism actually you should try it sometime

3 June at 12:11 | Open on circumstances.run
Kaye

@mozilla Good to know. Is there going to be a settings toggle to disable AI features altogether?

3 June at 10:01 | Open on octodon.social
Akseli :quake_verified:​ :kde:

@mozilla hm, i think this can be useful, however the problem is when people will never look at the output and just accept it at face value.

Basically I hope you will add a warning box that says "Do note that the text generation is not perfect and you should make sure the text clearly fits the image" or something along those lines. Also when it generates the text, it should always add "This alt text was generated by Firefox language model." as the first sentence, so people who rely on alt text features will know that this may be inaccurate.

@mozilla hm, i think this can be useful, however the problem is when people will never look at the output and just accept it at face value.

Basically I hope you will add a warning box that says "Do note that the text generation is not perfect and you should make sure the text clearly fits the image" or something along those lines. Also when it generates the text, it should always add "This alt text was generated by Firefox language model." as the first sentence, so people who rely on alt text features...

3 June at 10:03 | Open on scalie.zone
Rainer Zufall

@mozilla What is it actually trained on?

3 June at 10:05 | Open on mastodon.social
morgan

@mozilla this is not how we push for better accessibility. PLEASE reconsider this "AI" shift

3 June at 10:05 | Open on lingo.lol
modulux

@fay @mozilla This is additional accessibility, concurrent to whatever other push in whatever other direction can and should be done.

3 June at 11:51 | Open on node.isonomia.net
alexadeswift

@mozilla I absolutely do NOT want any AI embedded on any of my devices or applications for any reason. I hope therefore that any such thing will only ever be an add on to Firefox ?

3 June at 10:08 | Open on lgbtqia.space
DELETED

@mozilla

This is the first non dream stealing non tocic use of the clockwork parrots I have seen; good for firefox

3 June at 10:11 | Open on kolektiva.social
Greenpete (No Flag)
Fahri Reza

@mozilla waiting for the hilarity that will ensue

3 June at 10:16 | Open on mastodon.social
Danil

@mozilla cool.
Is it time to invent webbrowser that actually dont spy on you?

3 June at 10:19 | Open on mastodon.gamedev.place
lily 🏳️‍⚧️

@danil@mastodon.gamedev.place @mozilla@mozilla.social the web has been twisted specifically so you can't make new web browsers
try out ladybird browser. it's quite good! but there are many issues with it so you can't use it as a daily driver. all new browsers would have to be based on an old browser, which defeats the point of having more than one.

3 June at 11:59 | Open on possum.city
Latte macchiato :blobcoffee: :ablobcat_longlong:
maybeanerd

@mozilla would be amazing if this offered an API for webapps to use. E.g. mastodons alt text field could detect it has that feature available and provide a suggested alt text to users.

On the other hand this might encourage lower quality alt texts, as that will always be quicker to do than writing down your own alt text.

Maybe keeping it "fallback" only for consumers of content that is missing alt text is best.

3 June at 10:24 | Open on bumscode.com
MarjorieR

@maybeanerd @mozilla well we do already have OCR alt-text generation on Mastodon.
I do use that eg to capture the words in the titles for graphs but it requires heavy editing to get rid of the 'junk' (axes numbers, etc.) and to explain what the graph 'says'.
Would ML be able to really do the additional analysis and summarise it?

3 June at 10:40 | Open on social.linux.pizza
DELETED
Michal Bryxí 🌱

@mozilla AI haters in your replies in 3 ... 2 ... oh nevermind, they are already here 🫠

3 June at 10:26 | Open on veganism.social
Kai und der Andere

@MichalBryxi name one advantage of sabotaging accessibility. Just one. @mozilla

3 June at 12:31 | Open on antifa.style
Michal Bryxí 🌱

@walsonde How do you mean? How does providing an ability to get _some_ sense of information to group of users make the situation worse than giving them _no_ information?
I'm not going to ever argue that quality, thoughtful and well done descriptions made by humans won't be better. But that's not the option we have here on the table nor the _realistic_ situation for the users.

3 June at 14:22 | Open on veganism.social
Afferand

@mozilla Seems like a push in the right direction.

3 June at 10:28 | Open on mastodon.social
MarkAssPandi

@mozilla Ok I'll admit this is pretty cool

3 June at 10:38 | Open on gejuchowo.pl
Justin

@mozilla please ensure there's an option to turn it off. I don't want any AI in my system.

3 June at 10:39 | Open on toot.io
joene 🏴🍉🌲

@mozilla I don't want 'AI' on my computer! How to get rid of this 200MB?

3 June at 10:44 | Open on todon.nl
Emma

@mozilla cool. my visually impaired audience will be so happy i clicked the button to generate a bunch of gibberish, instead of spending 10 seconds to write a thoughtful description.

3 June at 10:44 | Open on mastodon.social
laura rachael

@mozilla no. get this garbage out of here

1. what the hell is it trained on? the same corpus of implicit bigotry and medical misinformation as every other "ai"?
2. it's just going to lie a bunch like every other "ai". wrong alt text is worse than no alt text.

3 June at 10:52 | Open on cyberpunk.lol
laura rachael

@mozilla when i was fresh out of college, you were the only tech company that still seemed good, and i hoped to work there someday. im glad i didnt; i would be ashamed to have your name on my resume.

ai.

are you fucking kidding me.

3 June at 10:58 | Open on cyberpunk.lol
Kaito
Kaito

@mozilla ok seriously though. One downside I can see from this is that this would blow up Firefox's installation size. A lot of people use Firefox as a browser for low-powered computers (think IoT / Rasp Pis). Bloating its installation size would make it a less attractive option. This would be better as an extension instead of a built-in feature. At least in my opinion. If Mozilla isn't considering this option then _surely_ there must be a compelling reason, right?

3 June at 13:26 | Open on mastodon.social
DELETED

@mozilla Will the AI be an always-on thing, or can it be turned off?

3 June at 11:05 | Open on stranger.social
Condalmo.

@mozilla Or, I don't know, people could type them? On the things they post? "This way, people don't have to think" - ?

3 June at 11:09 | Open on mstdn.social
Kai und der Andere

@mozilla you really make it hard for decent people to not switch to other browsers for good. At this point there's no other explanation than your managers being bribed by competitors.

3 June at 11:11 | Open on antifa.style
Echo 🦊

@mozilla please make sure that there is a opt out option for this feature, I do not want to use AI to generate alt text that can be potentially incorrect or misleading to the people that needed most, i’d say this feature is highly unnecessary while other features should be prioritized over adding more AI to a web browser. If I wanted AI in my web browser, I would be using Google Chrome and using glue to make sure the cheese doesn’t fall off my pizza.

3 June at 11:29 | Open on defcon.social
Tushar Chauhan
excited for the mastodon rise
surchaw
Zransgender (also known as "X During Pride Month")

@mozilla
Context is important though. AI (rather, ASI) will fail to understand the intent of a picutre. That's an easy way to render a person unable to understand the picture even with a physical explanation...

3 June at 11:53 | Open on furry.engineer
TrackerRoo :verified:

@mozilla So now alt text will be randomly generated gibberish. That sounds helpful. Hope AI bankrupts your company. Just stick to being a browser. It's all we've ever wanted from you. No crypto, no AI, just browser

3 June at 11:53 | Open on meow.social
T.J. Crowder

@mozilla This is cool-ish (and love the private, on-device aspect), but seems like the wrong end of the pipeline. Does Firefox help people write their alt text when posting images (and prompt them to do it)? Now _that_ would be game-changing. No need for the cost of the power increase of hundreds or thousands of _readers_ to create it, do it when the _writer_ creates it.

3 June at 11:59 | Open on hachyderm.io
Chuck Munson

@mozilla I've been a loyal Fireox user since the browser was launched and a web developer since that time. I don't want AI in the software I use and will switch when that crap is added. I guess I'll have to use Vivaldi more. This is disappointing news. Nobody wants this garbage in their browsers.

3 June at 12:06 | Open on breadandroses.cloud
Apan

@chuck @mozilla Nobody? That's a pretty bold claim. Any source for that claim. Maybe it's just your opinion?. It's probably very easy disable in the about:config.
My opinion however is pretty indifferent to this feature. I would prefer mozilla to focus more on privacy and security.

3 June at 12:39 | Open on mastodon.social
GrayGooGirl :v_lesbian: :1up:

@mozilla
Hard fucking pass. I don't want a tiny LLM using my device resources.

3 June at 12:11 | Open on mstdn.games
Alexander 😷

@mozilla Please stop with the "AI" bullshit.

3 June at 12:14 | Open on tech.lgbt
Danielle Pond

@mozilla I would have preferred it if you had just focused on speed. I don't need AI built into every piece of software I use.

3 June at 12:20 | Open on mastodon.gamedev.place
DELETED
hapax

@mozilla no no no. No AI in Firefox, PLEASE. Y'all are the one viable alternative browser, please don't start following silly trends like this. If people want AI nonsense, there's plenty of places to get it. I would happily pay to avoid having features like these creep into Firefox.

3 June at 12:21 | Open on discuss.systems
Ænðr E. Feldstraw

@mozilla So, erm… how often will it phone home to teach a centralized model about how to do this? How much are you going to pay your laborers for their time to train your a.i. models?

3 June at 12:25 | Open on mastodon.social
skua

@mozilla
What a shitty idea.

Wish I was surprised that Mozilla has jumped on the AI Highway to Hell.

But their priorities have not been user focused for years IMO.

youtube.com/watch?v=4hhlQU0zDp

3 June at 12:27 | Open on mastodon.social
Martin Rocket

@skua Why shitty? It's done locally, it doesn't share information with an external provider. It does improve user experience.

3 June at 13:49 | Open on det.social
skua

@Rocket
UX is improved by inter-human communication.

AI alt-text obstructs and supplants interhuman communication.

UX is improved by alt-text that communicates the poster's interpretation of the image.

AI alt-text makes it easier for the poster to not share their interpretation.

Seeing a diminishment of UX when Mozilla's AI alt-text generator is used.

3 June at 14:32 | Open on mastodon.social
Martin Rocket

@skua I totally see your point.

I'd say it's a matter of perspective. If you compare AI alt-texts and author supplied alt-text, then the AI alt-texts are clearly the worse user experience.
But if the alternative is not having alt-texts at all, then AI alt-texts are better.
Or if the alternative is relying on a central AI service for alt texts, then local AI alt text generators are better.

3 June at 14:49 | Open on det.social
Schneckbert 🐌

@mozilla You can keep anything AI away from me as far as possible at this point. Invest your time on something more useful - like watching paint dry or count sand on the beach.

3 June at 12:33 | Open on chaos.social
Scott D. Strader

@mozilla The amount of kvetching in the comments here is hilarious. I guess it should be expected if you combine AI, open source, and alt-text into a single post on Mastodon.

3 June at 12:37 | Open on masto.ai
Methuselah
Lucas Gonze

@mozilla using AI to fill in missing alt text is an excellent application, however that AI is trained on alt text, so in the long run there will be less and less training data and the AI will get wronger and wronger.

3 June at 12:38 | Open on indieweb.social
lunya (cute) :neocat_floof:

@mozilla@mozilla.social yayy :3

This is good

But please

Please, do things other than ai

3 June at 12:39 | Open on fearness.org
Dream Hollow

@mozilla Automated alt-text isn't too bad. This is a fair use of AI that doesn't really step on any toes.

I could foresee this causing problems if the alt-text is very wrong, though.

3 June at 12:44 | Open on mastodon.social
DELETED

@mozilla "AI" as it exists currently is so shaky and bad that adding it to anything is just a bad idea.

3 June at 12:50 | Open on meow.social
Syphist :verifiedtrans:

@mozilla@mozilla.social why does my browser need AI at all? Can we stop wasting electricity on this snake oil?

3 June at 12:50 | Open on zoner.work
filobus

@mozilla "The first time the user adds an image, they’ll have to wait a bit for downloading the model (which can take up to a few minutes depending on your connection) but the subsequent uses will be much faster"

Hope you can disable this feature and 200 mb download at all
I understand it can be very useful for some users, but for me not at all
If I find porno images I don't understand I think I can skip them 🤪

3 June at 12:55 | Open on sociale.network
ikt 🇺🇦

@mozilla ITT all 7 firefox users upset that AI might be useful for some things

Not pictured: 2 billion people using Chrome

3 June at 12:58 | Open on fosstodon.org
Denis Drakhnia
@mozilla

That time has come...

We need to form a union to protect Fediverse users from losing their jobs creating alt-text for images!
3 June at 12:58 | Open on suya.place
Florian

Alt text will absolutely still be required on websites and social media posts; this is just to patch over where people couldn't be bothered to be inclusive, basically :)

3 June at 13:09 | Open on cupoftea.social
Inara :therian:

@zersiax or are unable to use english to describe things and can easily get overwelmed by it and cause the vessel freeze up and space out like me

3 June at 13:31 | Open on meow.social
Florian

@Kazhumazu or that :) fortunately there's usually hashtags to request others to do this for you if you do run into that issue, web is harder, but using an AI you might be able to have it generate the alt and then edit as needed. I'm cognisant of your struggles, I really am, but as a member of an audience for whom the difference between being included in a conversation or not depends on alt text being there or not I also have to be somewhat firm there. There is ways, people just refuse to use them en masse and unfortunately yank people like yourself who just have a lot of trouble with it into the crossfire and I'm truly sorry about that :( I appreciate you responding!

@Kazhumazu or that :) fortunately there's usually hashtags to request others to do this for you if you do run into that issue, web is harder, but using an AI you might be able to have it generate the alt and then edit as needed. I'm cognisant of your struggles, I really am, but as a member of an audience for whom the difference between being included in a conversation or not depends on alt text being there or not I also have to be somewhat firm there. There is ways, people just refuse to use them...

3 June at 14:10 | Open on cupoftea.social
sll - Macron destitution.

@mozilla
A clever way to make developers and site owners not forget to write proper alt text. "Write your images alt text, otherwise we'll generate gibberish and deface your websites!".
Smart move, Mozilla.

3 June at 13:10 | Open on pouet.chapril.org
Site Reliability Enby🏳️‍⚧️🏁🔦📈🐺👗

@mozilla How much RAM and CPU does this cost?

3 June at 13:22 | Open on tech.lgbt
Dźwiedziu

@mozilla I'm still waiting for the feature that obligatory speaks back the comment to the user before posting.

(see xkcd.com/481/)

3 June at 13:27 | Open on mastodon.social
Benjamin Kwiecień 🇵🇸
Harrison Totty

@mozilla This is a good use of the technology.

3 June at 13:38 | Open on mastodon.social
Min

@mozilla@mozilla.social I appreciate the effort to make the web more accessible, but:
1. How do we know this model was trained ethically? Literally every human text LLM I've seen was trained on data that was used without permission of the author.
2. What quantitative testing have you done to ensure that the results of this model are correct? A model that produces incorrect descriptions a significant percent of the time is worse than not having one at all.

3 June at 13:43 | Open on fuzzyfox.social
Copernicron

@mozilla will I be able to turn it off? I don’t want AI everywhere and in everything.

3 June at 13:45 | Open on mstdn.party
Martin Rocket

@mozilla I asked ChatGPT for a short prompt for the candle picture:
“The image shows a birthday cake with lit candles in the foreground and a smiling woman in the background, likely in a room with several people.”
That's indeed longer than the Firefox text but not absurdly lengthy and detailed.
However, I'm impressed that Firefox does that locally.

3 June at 13:47 | Open on det.social
Pier-Luc Brault

@mozilla Sounds like a positive and non-intrusive integration of AI in an application for once!

3 June at 13:52 | Open on fosstodon.org
Kevin Karhan :verified:

@mozilla NO GOD PLEASE NO!!

- Instead of doing #BuzzwordDrivenDebelopmen.and #HypeBasedDevelopment, FOCUS ON MAKING FIREFOX BETTER FOR ONCE!

Seriously, nobody asked for this!

- Of all the stuff you could've added to #Firefox, you choose #AI bs instead of fixing bugs, issues and integrating actual #Privacy and #Security features, like native support for #Tor, #I2P or even #IPFS instead...

3 June at 14:15 | Open on infosec.space
Lucas Gonze

@kkarhan This is kneejerk anti-LLM. The feature is clearly making Firefox better for people who rely on alt text.

3 June at 14:17 | Open on indieweb.social
Nobody باچیز नास्ति (he/him)

@mozilla

How much energy will that consume?

3 June at 14:33 | Open on mastodon.social
OddOpinions5

@mozilla

People

I have a really radical idea

Please don't flame me until you read it !!

Lets see how this works for a week or so before we decide if it is good or bad

Could be that 50% of the time this feature works, which would save everyone a lot of time

3 June at 14:34 | Open on mas.to
Carlos Solís
Warning: this model is derived from WebText, GPT-2's proprietary training data corpus, so the same copyright caveats apply. github.com/openai/gpt-2/blob/m…
3 June at 14:39 | Open on hub.azkware.net
DELETED

@mozilla Mozilla is long gone, support an actual independent effort for the open web palemoon.org

3 June at 14:43 | Open on mastodon.social
Itchy
Marcus

@mozilla This is pretty cool; but I was unaware that Firefox has a built in PDF "editor" and not just viewer.

3 June at 14:48 | Open on mastodon.social
Osma A

@mozilla
Still waiting for on-device text translation for mobile devices.

3 June at 14:48 | Open on mas.to
Wyatt (🏳️‍⚧️♀?)

@mozilla Cool, now make it work on older CPU's if we want to let it be slow

3 June at 14:55 | Open on tech.lgbt
Not David Beckham
Simon Lucy
F4GRX Sébastien

@mozilla how can I disable it? I dont have free gigabytes for this shit.

3 June at 15:02 | Open on chaos.social
Janie Karma S 🏳️‍🌈🖖🏿
Mmmm

@mozilla Nice! Your PDF functionality was neat, so this looks promising!

3 June at 15:08 | Open on mastodon.social
Essem :skeeter:

@mozilla very interesting approach to the issue, while i probably won't be using this regularly myself i know some people that could find this useful

props for taking the extra effort to make it fully local as well

3 June at 15:15 | Open on wetdry.world
Petesmom

@mozilla literally moved to Firefox to get AWAY from stoopid AI hype. WHYYYYYY????!!!!

3 June at 15:37 | Open on mstdn.party
hacknorris
Noble :frostbite:

@mozilla thank GOD they're actually using AI for something useful and doing it in a privacy-respecting way rather than just slapping a cloud-based LLM on the side of the browser and calling it a day like pretty much every other browser does
i'm sure for the visually impaired having something to describe an image is better than nothing at all. i hope that the generated alt text specifies that it was automatically made though so that it isn't interpreted as 100% accurate

3 June at 16:20 | Open on wolfdo.gg
boingo-00 🦊 :fluffytail:
@mozilla if it works like offline translator, it is good
3 June at 16:22 | Open on pleroma.boingo00.wtf
Freeeeeeeeeeeeeeeeeeeee :ablobfoxbongohyper:‮‮‮‮‮‮

@mozilla i hope it is noticeably mentioned which are generated and that it can be inaccurate at least

3 June at 16:32 | Open on wetdry.world
Steven Goetz 🇨🇦
srg
`Da Elf

I don't actually Want that in my browser.

If I did, I suppose I'd run, I don't know, Fucking Windows?

3 June at 17:48 | Open on mstdn.social
Spooky Guy
Hunterrules

@mozilla Im ashamed to use this shitty browser that keeps getting worse. at this point the only way to save firefox is to fire all the devolpers who make these changes and never let them work on anything again except those little "square hole" baby toys.

3 June at 19:08 | Open on techhub.social
Hannah Kolbeck 🏳️‍⚧️

@mozilla I've been working on tools around issues like this for years, including a large amount of communication with folks who depend on alt text. Should this technology be available to those writing alt text, rather than just voluntarily to screen reader users, it will be immensely harmful to the accessibility of social media everywhere. Please do not do this. I'm happy to dive into why but please do not make it easy for people posting images to use this AI generation.
- Founder, Alt-Text.org

3 June at 19:28 | Open on social.alt-text.org
Jamie is a friendly nut

@mozilla this looks like an awesome accessibility feature! thank you for including an easy-to-understand rundown in the article of how this tool differs from the ethically-dubious LLM stuff that's everywhere nowadays

3 June at 19:42 | Open on mastodon.social
יואב זק :yellow_ribbon:

@mozilla
maybe make it opt-in? this is the best course of action for any controversial additions, and you must know it is controversial.

3 June at 19:46 | Open on tooot.im
Alex Holst

@mozilla Funny. Just today I was commenting on how Firefox only survives now because of Manifest V2.

Every release takes you further and further from what users actually need.

3 June at 20:11 | Open on helvede.net
Javielico 🌖

@mozilla I've serious concerns about this. I get that most likely other browsers will do it, but I feel like if we let people generate alt captions via AI models the people who will benefit from those captions will most likely miss out of the details that only a human looking at an image will actually see and be able express? If anything the AI model will make things up and explain things in such detail that might not be relevant to the actual image/scene itself.

3 June at 20:18 | Open on social.coop
james

@mozilla whilst I admire the desire to make accessibility easier and more present, instagram and similar have auto alt text and it is absolutely rubbish. The most famous users of it do not check it at all. My favourite artists will allow “possible three panel illustrated graphic comic of a green snake or reptile having a coversation” alongside their image

We must fund alt text education and have suitable controls and alerts as to the usefulness of the generated content, alongside bringing these “features” into the world.

@mozilla whilst I admire the desire to make accessibility easier and more present, instagram and similar have auto alt text and it is absolutely rubbish. The most famous users of it do not check it at all. My favourite artists will allow “possible three panel illustrated graphic comic of a green snake or reptile having a coversation” alongside their image

3 June at 20:24 | Open on strangeobject.space
Luna Lactea

@mozilla This is good, but I feel like this shouldn't be a long term solution. Instead we should be pushing websites to serve alt text, especially since HTML already has support for it.

3 June at 21:20 | Open on furry.engineer
Go Up