|
Top-level
 So here is Paul Baran's "literally the most influential paper to affect networking systems ever" 1964 paper: "On Distributed Communications: I. Introduction to Distributed Communication Networks" https://www.rand.org/pubs/research_memoranda/RM3420.html It's good, it's amazing, it's INCREDIBLY visionary 79 comments
And just imagine the *gall* of it: telling the *military* let alone the world oh you know how you love hierarchy? Well guess what, you know what's WAY better, something that's closer to cooperative anarchy, where there's a lot of cooperation lots of error-prone little guys AND HE WAS RIGHT Baran comes in with the math to back up his claims, a vision of how basically wifi and satellite and land lines and cable internet would all work together before we even *had* any internet stuff, shows how a packet would look, and says if you want to REALLY be tough, be... "distributed" Hm, did you notice I said "distributed" and not "decentralized"? Actually wait... does this sound familiar, have you heard of this paper before? Could it be? No... it couldn't be... And yes of course it is literally the paper that gives us this incredible FIGURE 1, which you have CERTAINLY seen if you have ever heard ANYONE talk about ANY "decentralized" or "distributed" system ever CENTRALIZED DECENTRALIZED DISTRIBUTED You know this image. You could never forget this image One of the reasons you know this image is that everyone worth their salt who works on decentralized networks thinks about this image and puts it in their talks But also so does this bro who has literally no idea about how tech works but thinks he does So one way or another you're gonna see it (tech bro courtesy https://www.threepanelsoul.com/comic/job-interviews) That comic is from Three Panel Soul btw, and here's the link https://www.threepanelsoul.com/comic/job-interviews All of Three Panel Soul is good, but the Tech Bro ones are my favorites https://www.threepanelsoul.com/comic/search/Tech%20Bro I love Three Panel Soul so much (Gonna weird out @3psboyd by fangirling over here) *COUGH* where was I "Christine if you love this paper so much why don't you like the definition of 'decentralized' from it?!" The definition is great actually if you know the context Because the context is CRITICIZING THE DESIGN UNDER THE DEFINITION AS A FORM OF CENTRALIZATION "What Christine you can't mean that, why would 'decentralized' be 'centralized' that can't be true" Because because BECAUSE my good friend, Baran was describing "decentralization", a term that ALREADY EXISTED in networking, as being a kind of centralized system NO REALLY I AM SERIOUS The term "decentralized" was *already* in active use! So Baran was providing "distributed" as the new term! Oh my god THAT'S WHY THE DEFINITION BARAN PROVIDED FOR DECENTRALIZATION WAS SO WEAK You don't believe me? Let me show you. LET ME SHOW YOU Here is where Baran defines "decentralization!" We have to read the whole definition! > The centralized network is obviously vulnerable as destruction of a single central node destroys communication between the end station. (cotd) Baran "decentralization" cotd: > In practice, a mixture of star and mesh components is used to form communication networks. IN PRACTICE FOR CENTRALIZED SYSTEMS YOU GUYS (cotd) Baran "decentralization" cotd: > For example, type (b) in Fig. 1 shows the hierarchical structure of a set of stars connected in the form of a larger star with an additional link forming a loop. OH SHIT HE'S STILL TALKING ABOUT CENTRALIZATION FIGURE B IS THE MIDDLE ONE (cotd) Baran "decentralization" cotd: > Such a network is sometimes called a "decentralized" network, because complete reliance upon a single point is not always required. OKAY WE'RE DONE But look at it all together! He's talking about how "decentralization" is a term of art but it's still CENTRALIZED Baran didn't make up the term "decentralized" it already was being used in practice to talk about top-down hierarchical systems! Baran calls this version centralized even if there's a "loop" (a small number of top-level providers)! YOU GUYS THIS IS NOT HOW WE ARE USING "DECENTRALIZED" WE are not describing the future of routing small packets in 1964, that is NOT the world we are existing in, where "decentralized" meant a top-down hierarchical structure When WE talk about "decentralized", we mean roughly a spectrum, with "centralized" on one side and "decentralized" on the other Now I don't think Bryan Newbold realized that when he pulled his definition from Mark Nottingham who pulled his definition from Paul Baran, that this was the case. I think this is a game of telephone. (I don't know how Mark Nottingham didn't realize it but that's an aside) What I DO know is that it means that the entire structure of analyzing decentralization in Mark's paper and Bryan's blogpost thus, in practice, surround a term that is weak because it was FUNDAMENTALLY describing a centralized system, so it could criticize it The loss of context here is BRUTAL To conflate the two *automatically* introduces decentralization-washing. I don't think this is intentional, but it explains a lot. It explains how a "weak" definition of decentralization could come from one of the boldest visions of what that very *idea* could be Now okay let's point out the irony here because I feel like if I don't I'm being mean. Bryan does say: > To some degree, I don't really want to spend time in a terminology debate. And I just did! At length! But the whole debate this whole time is "is Bluesky decentralized" so we kinda HAVE to But also what happened was: - I lay out a strong definition of decentralization; Bluesky doesn't match And I don't think this was malicious on Bryan's part in the least because I know Bryan well enough to know he's not like that! I am pretty annoyed at Mark though for quoting this out of context in such a way that it can completely confuse a narrative like this. I'll assume that was a mistake but The reality is that Bluesky didn't match my definition of decentralization, and I hope it's pretty clear now that the alternate definition supplied was literally one about centralization And so that cannot possibly be a lower bar that we say "okay maybe Bluesky can pass this one" I am sorry Let's PLEASE not move the goalposts on "decentralization". Let's certainly not move them back to something that was literally "here's what centralization looks like in practice". That's what I'm asking for here. That's why I went so goddamned HARD on terminology here. Let's check the time. It's 7:30pm where I am. I woke up at 4:30am and resumed work on my blogpost at 5am. I have been, for the most part, between the blogpost, my job, and this thread, sitting at my computer fighting for decentralization for about 14 hours. It's been like that a lot lately. I have a reputation at work of being good at pushing others to take off time and they HAVE to take off time OR ELSE and I try to be that way in general. But I am really truly bad at doing so for myself and I know I have crossed my limits for today. So let's wrap up for *tonight* in a sec We're about halfway through this blogpost. There's a lot going on in my life. I am trying so hard to keep the organization I work for alive and moving forward. I am tired. I need rest. And I still need to drive two hours across the state tonight. We're going to resume tomorrow. But first... There's a reason I'm going really hard on this. I really care a lot about the shape of the internet. And tomorrow we're going to get into some more analysis and a talk about *values*, and one thing I like is that Bryan talked at length about Bluesky's values. And I think that part was really good. For tonight, I need to unwind, I need to put a label on a mailbox, I need to eat dinner, I need to drive across the state, I need to sleep. Maybe I appear ridiculous. I get it. I go pretty hardcore on this stuff. If you know me you know I tend to go all in. I am signing off for the night. Tomorrow we will analyze whether or not my assertion that "ATProto has explosive behavior as it approaches decentralization" problems. I'm not going to read notifications until I finish this. Maybe someone will prove me wrong before I get it done. I'll be oblivious. We will also analyze values, which maybe I care about more than anything. And there will be more secret goblins, hidden among the posts. For tonight, it's rest time. It's time for a === NO MORE LOOKING AT MY COMPUTER BREAK === Hello! I am back at my computer. Today we are going to talk about how ATProto does in terms of scaling. Yes, we know it scales up, and has done an impressive job of doing so! But what about scaling towards decentralization? Does it scale down? And does it scale wide? Let's look. Before we get deep into that, when we left last night I was extremely tired and had been working at my computer for over 14 hours. I then said I was going to drive two hours across the state that evening. Thankfully thanks to the support of people who love me, I did not do that foolish thing! So anyway, I am better rested, and also I woke up to the surprise that our fundraiser is doing a lot better, like by a lot, than it was yesterday, which is nice because I was extremely stressed out https://spritely.institute/donate/ So I am feeling much better and alive and today I remembered to eat lunch But you probably aren't here to hear about my lunch choices or how much sleep I got or whether or not I forgot to bring my ADHD medication with me (I did so now I am drinking a bunch of caffeine instead), you are probably here to hear the rest of the analysis about decentralization and Bluesky etc So let us get to it, let's talk about whether or not Bluesky can scale *down* in a meaningful way. In my last essay I made assertions that this was important for decentralization and said ATProto wasn't great for this, and this was one thing people challenged me on So let's take a look! When I say "scale down", what I generally mean is "small instances can generally participate on the network". (We'll talk about "scale wide" later.) But another useful possibility which has come up is "can you make a smaller, more isolated use-case and use the same protocol for it" This latter version of scale down does come up in Bryan's article: > A specific form of scale-down which is an important design goal is that folks building new applications (new Lexicons) can "start small", with server needs proportional to the size of their sub-network. (cotd) Strictly speaking, I agree, ATProto can scale down in this use case! For example, if you wanted to make a small specialized forum for collaborative storytelling, you could use ATProto for it, and that's true, you could do it But is it the right choice? In some ways we are talking about two different things here: extension of functionality (which you might want the same scale for) and having a smaller and more isolated community But regardless ATproto positions itself *specifically* as designed for not wanting to miss messages, and I talked previously about how ATProto's design requires a god's-eye view. It's a bit strange of a choice when you say "let's run a smaller community" Given that message passing systems handle small scale systems *beautifully*, and *still* allow for interactions with larger scale systems, it's a bit confusing to me *why* you'd choose ATProto for such use cases. What is the specific benefit you'd gain? Especially because it's actually lossier here At any rate, there's a bit of conflation here. "It scales down" by saying "you can have an isolated community/use case that's oblivious to the rest of the system" is categorically distinct from "it scales down" in terms of "a small node can meaningfully participate with the larger system" At any rate, the problem with "scaling down" is much clearer when it comes to the problem of "scaling wide". Or let me put it a different way: ATProto *explodes in complexity* when you try to scale it towards meaningful decentralization Yes that's right we're getting to the spicy part of this conversation. We did the warm-up, now it's time to talk about the real thing, whether or not decentralization in the way I believe people *think* that term means is reasonably possible with ATProto as it's currently designed But before we do that, I need to stretch and run to the bathroom So for those of you following along, if you found this, Secret Goblin #3, let me know: "👺" Oops wait actually we gotta talk about that one for a sec there's a reason I left it in scare quotes Why on earth is the textual descriptor for Unicode U+1F47A "JAPANESE GOBLIN", does anyone know? It's a Tengu, right? Despite being the only actually named "goblin" emoji, I feel awkward about this one because is it correct to call it a "JAPANESE GOBLIN" instead of just "TENGU"?!?! I don't know! If you have knowledge or OPINIONS about "👺", its name choice in unicode, or, for that matter, a white person just dropping it in the middle of a group chat WITHOUT putting it in quotes (I did tho), feel free to derail the comment thread Otherwise it's time for a === STRETCH BREAK === I'm back. It's time to talk about it: does Bluesky/ATProto suffer a "quadratic explosion" as we move from centralization towards *meaningful* decentralization? I claimed it did, but I was challenged on this. What did I mean? Am I right or wrong? It's time to find out! In the previous blogpost I said the following: > If this sounds infeasible to do in our metaphorical domestic environment, that's because it is. A world of full self-hosting is not possible with Bluesky. (cotd) Decentralized ATProto is quadratic quote, cotd: > In fact, it is worse than the storage requirements, because the message delivery requirements become quadratic at the scale of full decentralization: to send a message to one user is to send a message to all. Rather than writing one letter, a copy of that letter must be made and delivered to every person on earth. |
Gregory's Server
So okay yeah it's very military-oriented but... but! The context for this paper is that Paul Baran is arguing for what eventually *becomes* networking as we know it. Baran says: let's use *cheap* equipment with *way less centralization that we've ever seen* and it'll be *better actually!*