Email or username:

Password:

Forgot your password?
vruz

Stanford has created an AI model called Stanford Alpaca.
youtube.com/watch?v=xslW5sQOkC

It's capable of doing what ChatGPT does and more, except at a fraction of the cost: $600. They were allowed to do this because Alpaca didn't have to be trained same as GPT was, it only had to learn all of GPT's tricks watching GPT play.

Microsoft placed a $2 billion bet on OpenAI, so that makes them look $2 billion less smart than last week.
the-decoder.com/stanfords-alpa

#OpenAI #AGI #gpt #chatgpt #StanfordAlpaca

43 comments
vruz

@HistoPol It gets even better. All Apple, Google, and Amazon announced internal projects to counter OpenAI only this past week.

What if other universities and medium-sized upstarts create a hundred other Alpacas?

Will MAGAFA have to withdraw their precious APIs, so that other models don't learn from theirs?

I don't share the dread other people feel over these developments, the very big corporations are accelerating exponentially on a collision course with their own demise.

vruz

@HistoPol Also, there's a non-zero chance that Google are actually behind Stanford Alpaca. Everybody knows Google has always been a Stanford shop.

Samhain Night

@vruz So it doesn’t work, just as badly as the $2 billion Microsoft one. Got it.

Olav

@vruz given the ethics/discussion about how LLMs are trained, a hearty lol is in order.

Olav

@vruz nice one!

Stupid sans-serif fonts.

Kale

@olavf @vruz I'll happily bootstrap an AI by pirating someone else's ethically shady shit. Seems more ethical that way.

vruz

@DarkestKale @olavf

It's a tool. If the tool is –for example– used to ethically produce Free and Open Source Software, its byproduct could be more ethical than the way it started.

Of course, it could also be used to dramatically accelerate the production of disinformation.

Faced with these scenarios, I'm sure that the choices will begin to present themselves very clearly, starkly contrasted even.

Olav

@vruz
@DarkestKale if it's lifting routines published under cc:by without attribution, it's not so ethical.

vruz

@olavf @DarkestKale

That's not how Alpaca works.
Maybe watch the video.

Olav

@vruz
@DarkestKale I was referring to Chat-GPT, etc. Sorry if there's confusion

SocialistStan

@vruz Oh no! Oligarchs are losing control of their technology! Soon the masses will have it! How terrible! (sarcasm)

Brian Hayes

@vruz
I bumped into Simon Willison’s blog discussing Alpaca.

He coined a term for this period, calls it a “Stable Diffusion moment” and says things accelerate the [very] moment this stuff is available for people to experiment with. simonwillison.net/2023/Mar/13/

vruz

@thoughtshopper Interesting. Will read later. Thanks for sharing.

Things in space and time

Hah! It really *is* plagiarism all the way down.

@vruz

OpenDNA⚙️

@vruz I know the founder of a profitable AI company.

Their "moat" is providing an extremely valuable service to a clearly-defined market of industrial customers with deep pockets and physical-world problems.

vruz

@opendna For people who are proponents of capitalism as it was intended, that's a best case scenario.

Of course, these days in centrally planned capitalism, that person is in the vast minority.

OpenDNA⚙️

@vruz It has occurred to me that the early investors in OpenAI &etc may well know that there's no business model for ChatGPT and Stable Diffusion etc, and that's why they're pumping the PR so hard: the only viable exit is an IPO and Greater Fools.

If they actually believed the prospects matched their hype, they'd be stealth af.

vruz

@opendna

Counterpoint: I don't believe they are anywhere near so smart.

mmby

@vruz so, what we'll see is large companies trying to protect their models via legislation while extracting all the public and non-public sources for every ounce of marketable training data they can get

vruz

@mmby Fast enough that they can erect new barriers to entry. But fast enough, in this case, would have to be a matter of hours, which just can't happen in the legal world.

If that doesn't happen with the speed required, thousands of medium-sized upstarts have enough time in the coming days and weeks to attempt to reproduce what Stanford did.

Peter Zingg :verified:

@vruz And Stanford’s billion-dollar legal team can save money by using Alpaca to write the defense briefs for the next ten years of court proceedings in OpenAI vs The Leland Stanford Junior University.

vruz

@pzingg It's a tall order call for anyone to punish Stanford, who are not reproducing literally any protected intellectual property works, nor their technical methods.

I think the real legal problem is for MAGAFA to erect new legal barriers to innovation fast enough that they can even begin to make a profit from this type of technology.

Merovius

@vruz @dr2chase The irony of course being, that ChatGPT itself was trained by appropriating other people's work without asking or paying. So they should have a hard time arguing that this is unethical/illegal.

vruz

@Merovius @dr2chase

Alpaca also borrows quite a lot from Meta's open sourced training materials, so, yes... in a way it's indeed coming back at them.

Not that this is qualitatively an unexpected aspect of Free and Open Source software, what's unexpected and disruptive is the dramatic, possibly unprecedented magnitude of the drop of the cost of producing a comparable product from openly available materials. Near zero, as we discuss it, which wasn't supposed to happen for another few years.

dr2chase

@vruz @Merovius I am mystified that it works this well, I suppose I could read the paper but I am not sure I would understand it. Inferring eleventy-zillion parameters on the cheap, how does that work?

(Though I suppose you could iterate and search, something like that.)

I also wonder, suppose you fuzzed all the parameters, just-a-bit, what does that do to the output?

Bindestrich

@Merovius I find it amusing that none of the OpenAI people seem to have read dürrenmats the physicists. If so they would not do a surprised Pikachu now

Corbin

@vruz Ignoring the cult leader that you're quoting, you are witnessing the standard story for a new mechanical improvement to an existing process.

The first few movers must invest quite a bit of money, and then the technique becomes well-known enough that further improvements are croudsourced. In the olden days, this is what patents were meant to ameliorate.

The real insight is that -- like with all software -- eventually the Free Software solution will be preferable, and the profit window will narrow to nothing.

@vruz Ignoring the cult leader that you're quoting, you are witnessing the standard story for a new mechanical improvement to an existing process.

The first few movers must invest quite a bit of money, and then the technique becomes well-known enough that further improvements are croudsourced. In the olden days, this is what patents were meant to ameliorate.

vruz

@corbin The real insight is that I managed to condense a highly technical topic in layman's terms that anyone could understand in just a couple of paragraphs so that everyone could reach their own conclusions, even you.

I agree I did a pretty darn good job, indeed, thank you.

Aron Pilhofer 🍩

@vruz I also am going to go on a limb here … buy would love to know whether its development was heavily subsidized through the labor of a cadre of underpaid graduate students.

vruz

@pilhofer You may be able to read the paper that's referred to from the linked sources. The names of the authors are written in latin characters in the paper.

Bindestrich

@pilhofer anything happening at any University is , from my experience.

Will Tuladhar-Douglas

@vruz

@artificialphilosopher

The use of the term "shoggoth" for an LLM is excellent here.

vruz

@yetiinabox @artificialphilosopher

We're right there, roaming about he valley at the foot of the Mountains of Madness. 🙂

vruz

@yetiinabox @artificialphilosopher

However I must confess that I was initially drawn to the naming of Stanford Alpaca and I quietly kept the appearance of a stern adult whilst still laughing inside like a silly teenager 😂

LLaMA ----> Alpaca

Will Tuladhar-Douglas

@vruz

@artificialphilosopher

Saw that. Made me wonder how forced the acronyms will be for the rest of the camelids...

vruz

@yetiinabox @artificialphilosopher

At this time I can't see them going all the way to Guanaco and Vicuña, but you never know!

Will Tuladhar-Douglas

@vruz

@artificialphilosopher

"Dromedary. When you have to cross the icy wastes of a collapsing world with a heavy shareholder profit burden, and have nothing meaningful to say."

Go Up