Email or username:

Password:

Forgot your password?
Darius Kazemi

I was asked during Q&A of the #PlatGovNet panel I was on yesterday about the energy consumption of many federated servers vs the implied efficiency of a centralized service in a data center.

But centralized services have user surveillance and advertising mechanisms that small community servers do not - my bet is that hundreds of thousands of smaller servers, hosted individually, and delivering only the services their communities WANT will ultimately do less stuff and thus use less energy.

15 comments
Shay Pierce

@darius seems like Ethereum is on the right track then? Everyone with a little staking box in their closet at home, processing a tiny bit of a distributed and encrypted meta-OS that the world can reliably run on?

𝓻𝓻π“ͺ

@darius @mistertim did a back-of-the-envelope calculation on the energy use of Twitter's timeline mixer: assemblag.es/@mistertim/110123

If we have some data from mastodon TL generation we could attempt at a comparison. I think you are right but the devil will be in the details, what assumptions we make and how we scope what falls in and out of that calculation

ToroidalCore

@darius You have to consider as well the diversity of size and hardware the instances run on. Some are larger, but something like a small instance for a couple dozen people or even a single user could be on a VPS with a bunch of other hosts. Or, if it's a small instance running on a home server, it could very well be doing other things.

The hardware my testing GoToSocial instance is running on does several other things, for example.

matdevdug

@darius Seems unlikely tbh. In 2009 Twitter had about 35 million DAU, so somewhere between 3-5x the current mastodon traffic. We know from this period Twitter was a relatively standard Rails app that looks like the attached image from their slide deck.

Since the Fediverse burns a ton of network traffic on communication between servers over the public internet (vs a local network inside of a single datacenter or two datacenters with an MPLS or fiber between them) and every Mastodon server needs kind of a lot of resources to just start and host one user, my guess is you could run a centralized Mastodon for 10 million users and consume a tiny fraction of the electricity the current system does.

Scaling a system up from 1000 users to 5000 users or 500,000 users is a small amount of resources vs adding 50 servers hosting 10,000 users. There are perks to the Fediverse but energy efficiency ain’t it.

@darius Seems unlikely tbh. In 2009 Twitter had about 35 million DAU, so somewhere between 3-5x the current mastodon traffic. We know from this period Twitter was a relatively standard Rails app that looks like the attached image from their slide deck.

Since the Fediverse burns a ton of network traffic on communication between servers over the public internet (vs a local network inside of a single datacenter or two datacenters with an MPLS or fiber between them) and every Mastodon server needs kind...

allison

@darius i think you also need to take into account the nature and value of what the energy is doing. the goal is not to eliminate energy use, but to use energy on things that are actually beneficial. i think it's reasonable to hold that centralized social media is not beneficial, and that therefore energy used for it is always a waste

Josh "cortex" Millard

@aparrish @darius on top of that, the lack of focused political power and lobby budget and legal department etc. among a widely distributed citizen network of small servers means that if the energy cost becomes prohibitive or irresponsible at at time of scarcity, the solution will not be to e.g. bribe congress into tweaking regs to subsidize a corporation, pretend the problem isn't there, and externalize the costs on everybody else.

Josh "cortex" Millard

@aparrish @darius like, a conversation about how efficient centralized compute is at scale should probably also take into account how much leverage the owners of that centralized compute installation have over local and regional and federal governments. Nobody's gonna be fighting over who gets to give 10K small independent servers a municipality-hobbling tax break or free real estate. Scale's efficiency is often bought, not inherent.

Darius Kazemi

This point from @aparrish is so good I'm quoting it in full:

> i think you also need to take into account the nature and value of what the energy is doing. the goal is not to eliminate energy use, but to use energy on things that are actually beneficial. i think it's reasonable to hold that centralized social media is not beneficial, and that therefore energy used for it is always a waste

friend.camp/@aparrish/11014101

vruz

@darius @aparrish

Somebody confronted with the task of arguing against Allison would have the enormous task of demonstrating that:

A) Facebook is efficient and every joule they use goes to deliver value to their users and only in lesser degree to advertisers.

B) Most users of social media (centrally planned and otherwise) overwhelmingly demand advertising.

C) It's not possible to build a social network without advertising.

Good luck with that.

[DATA EXPUNGED]
alive

@darius some super back of the envelope math from some numbers i have from working at twitter a while ago suggests if you converted all twitter's compute resources into fedi servers it could pretty easily support the same number of MAU that twitter had on fedi. that doesn't take into account things like inefficiency in individuals running servers (although the fact that most people run them on VPSes probably makes that moot) or network bandwidth (not sure how RPC fanout compares to activitypub traffic, this probably depends on the degree of centralization we end up with), but the order of magnitude seems about right.

twitter ran in a highly inefficient way, not to mention massive amounts of compute spent on ML models for ads serving, timeline algorithm assembly, etc, so it's pretty easy to do better than that :p

@darius some super back of the envelope math from some numbers i have from working at twitter a while ago suggests if you converted all twitter's compute resources into fedi servers it could pretty easily support the same number of MAU that twitter had on fedi. that doesn't take into account things like inefficiency in individuals running servers (although the fact that most people run them on VPSes probably makes that moot) or network bandwidth (not sure how RPC fanout compares to activitypub traffic,...

⛧ esoterik ⛧

@darius there's also the issue of induced demand. compute is cheaper than it ever was but we're using more energy on it than we ever were. it's reasonable to question whether "efficiencies" of centralization will ever lead to net decreases of energy used versus "inefficient" small local/federated uses.

Shoq

@darius I was just reading your Patreon bio. When I first heard of ActivityPub, I imagined thousands of digital project leaders who sounded just like you, who already understand this unique opportunity to make "local" a thing in community online services. I was disappointed to find so few. Can you suggest where am I am likely to discover more? I am particularly interested in community referral, refdesks, and public, academic and special libraries in general.

alys

@darius seems like the same logic would argue for switching as many computers as possible to chromebooks or similar thin client designs.

DELETED

@darius I think decentralized is better, but I think you're underestimating the sheer energy usage of ActivityPub.

If there are 2 AP servers, they share info with each other. If there's 3, A shares with B and with C, B shares with A and with C, and C shares with A and with B.

If there's 1000 of them, there are 1000! connections. If there are 100000 small servers, there are 100000! connections.

Factorials grow faster than exponentials. Ludicrously fast.

Go Up