Get the best out of #git and #sqlite by having your database versioned in a human-readable format:
```
# Create a local git repo and a db file
$ git init
$ sqlite3 ./test.db
sqlite> create table test(id int primary key, value text);
$ git add test.db
# Set the diff format for *.db files
$ git config diff.sqlite3.binary true
$ git config diff.sqlite3.textconv "echo .dump | sqlite3"
$ echo '*.db diff=sqlite3' >> .gitattributes
# Do some stuff with the db
$ sqlite3 ./test.db
sqlite> insert into test values(1, 'a');
sqlite> insert into test values(2, 'b');
sqlite> insert into test values(3, 'c');
sqlite> update test set text = 'aaa' where id = 1;
sqlite> delete from test where id = 3;
# Check the diff
$ git diff
diff --git a/test.db b/test.db
index 9d6e6db..c9a7a08 100644
--- a/test.db
+++ b/test.db
@@ -1,4 +1,6 @@
...
CREATE TABLE test(id int primary key, text text);
+INSERT INTO test VALUES(1,'aaa');
+INSERT INTO test VALUES(2,'b');
COMMIT;
```
Get the best out of #git and #sqlite by having your database versioned in a human-readable format:
```
# Create a local git repo and a db file
$ git init
$ sqlite3 ./test.db
sqlite> create table test(id int primary key, value text);
$ git add test.db
# Set the diff format for *.db files
$ git config diff.sqlite3.binary true
$ git config diff.sqlite3.textconv "echo .dump | sqlite3"
$ echo '*.db diff=sqlite3' >> .gitattributes
# Do some stuff with the db
There are a few generalizations in this article, but it mostly nails my thoughts on the current state of the IT industry.
Why can we watch 4K videos and play heavy games in hi-res on our new laptops, but Google Inbox takes 10-13 seconds to open an email that weighs a couple of MBs?
Why does Windows 10 take 30 minutes to update, when within that time frame I could flash a whole fresh Windows 10 ISO to an SSD drive 5 times?
Why do we have games that can draw hundreds of thousands of polygons on a screen in 16 ms, but most of the modern editors and IDEs can draw a single character on the screen within the same time frame, while consuming a comparable amount of RAM and CPU?
Why is writing code in IntelliJ today a much slower experience compared to writing code in vim/emacs on a 386 in the early 1990s? And don't tell me that autocompletion features justify the difference between an editor that takes 3 MB of RAM and one that takes 5 GB of RAM to edit the same project.
Why did Windows 95 take 30 MB of storage, but a vanilla installation of Android takes 6 GB?
Why does a keyboard app eat 150-200 MB of storage and is often responsible for 10-20% of the battery usage on many phones?
Why does a simple Electron-based todo/calendar app take 500 MB of storage?
Why do we want to run everything into Docker containers that take minutes or hours to build, when most of those applications would also be supported on the underlying bare metal?
Why did we get to the point where the best way of shipping and running an app across multiple systems is to pack it into a container, a fat Electron bundle, or a Flatpak/Snap package - in other words, every app becomes its own mini-OS with its own filesystem and dependencies, each of them with their own installation of libc, gnutils/busybox, Java, Python, Rust, node.js, Spring, Django, Express and all? Why did we decide to solve the problem of optimizing shared resources in a system by just giving up on solving it? Just because we assume that it's always cheaper to just add more storage and RAM?
Why does even a basic hello world Vue/React app install 200-300 MB of node_modules? What makes a hello world webapp 10x more complex than a whole Windows 95 installation?
We keep repeating "developer time is more expensive than computer time, so it's ok for an application to be dead inefficient if that saves a couple of days of engineering work", but I'd argue that even that doesn't apply anymore. I've spent the last couple of years working in companies where it takes hours (and sometimes days) to deliver a single change of 1-2 lines. All that time goes in huge pipelines that nobody understands in their entirety, compilation tasks that pull in GBs of dependencies just because a developer at some point wanted to try a new framework or flavour of programming in a module of 100 LoC, wasted electricity that goes in building and destroying dozens of containers just to run a test, and so on. While pipelines do their obscure work, developers take long, expensive breaks browsing social media, playing games or watching videos, because often they can't do any other work in the meantime - so much for "optimizing for engineering costs".
How come nobody gets enraged at such an inefficient use of both computing and human resources?
Would you buy a car that can run at 1% (or less) of its potential performance, built with a process that used <10% of the available engineering resources? Then why do we routinely buy and use devices that take 10 seconds to open a simple todo app in 2023? No amount of splash screen animations can sugarcoat that bitter pill.
The thing is that we know what's causing this problem as well.
As industries consolidate and monopolies/oligopolies form, businesses have less incentives for investing engineering resources in improving their products - or take risks with the development of new products or features based on customer's demand.
That creates a vicious cycle. Customers' expectation bars lower because they get used to sub-optimal solutions, because that's all they know and that's all they are used to. That drives businesses to take even less risks and enshittify their products even more, as they know that they can get away with even more sub-optimal solutions without losing market share - folks will just buy a new phone or laptop when they realize that their hardware can no longer store more than 20 Electron apps, or when their browser can't keep more than 10 tabs open without swapping memory pages. That drives the bar further down. Businesses are incentivised to push out MVPs at a franctic pace and call them products - marketing and design tricks will cover the engineering gaps anyway. Moreover, now companies have even one more incentive to enshittify their product: if the same software can no longer run on the same device, make money out of the new hardware that people will be forced to buy (because, of course, you've made it hard to repair or replace components on their existing hardware). And the cycle repeats. Until you reach a point where progress isn't about getting new stuff, nor getting better versions of the existing stuff, but just about buying better hardware in order to do the same stuff that we used to do 10-15 years ago.
Note however that it doesn't have to be always like this. The author brings a good counter-example: gaming.
Gamers are definitely *not* ok if a new version of a game has a few more ms latency than the previous one. They buy expensive hardware, and they expect that the software that they run on that hardware makes the best use of the available resources. As a result, gaming companies are pushed to release every time titles that draw more polygons on the screen than the previous version, while not requiring a 2-10x bump in resource requirements.
If the gaming industry hadn't had such a demanding user base, I wouldn't be surprised if games in 2023 looked pretty much like the SNES 2D games back in the early 1990s, while using up 100-1000x more resources.
I guess that the best solution to the decay problem that affects our industry would be if users of non-gaming software started to have similar expectations as their gaming fellows, and they could just walk away from the products that can't deliver on their expectations.
There are a few generalizations in this article, but it mostly nails my thoughts on the current state of the IT industry.
Why can we watch 4K videos and play heavy games in hi-res on our new laptops, but Google Inbox takes 10-13 seconds to open an email that weighs a couple of MBs?
Why does Windows 10 take 30 minutes to update, when within that time frame I could flash a whole fresh Windows 10 ISO to an SSD drive 5 times?
@blacklight That article hits home indeed. I never understood why huge frameworks like Electron became popular and I sincerely believe it and tools that come along with it the web is going to shit nowadays. You can't just render a simple HTML page anymore, now you need to pull in giant JS frameworks that slow down your PC to a crawl.
I read an article a while ago that advocated for slowing down internet connections on purpose just like we have speed limits for cars. I really agreed with it.
@blacklight Fatal error: operation expectations.increase not implemented on User class. Really, how you can imagine a user would "they could just walk away from the products that can't deliver on their expectations."? There is single way for many users to go away from existing apps in many spheres - go offline, gadget-less, zero-IT, no-computers life, but this is a thing a 0.001% of enthusiasts can afford. Ordinary user often already can't do even this "f..k you" move.
The engineers who designed the #Voyager probes half a century ago even thought of the possibility that a wrong sequence of commands may point the antenna dish away from earth (like someone did a couple of days ago).
And they implemented a self-adjusting mechanism that a few times a year scans the positions of a few known stars to infer the position of the earth, and point back the antenna in the right direction.
50 years later, these wonderful machines are still working, tens of billions of km away from earth, with only 69 KB of RAM, and even a wrong sequence of commands won't put them out of use, while nowadays 4 GB of RAM aren't even enough to start VsCode or IntelliJ.
The more I understand how they were designed, the more I feel like an early Medieval engineer looking at the Pantheon or other marvels of Roman architecture. Some amazing skills, knowledge and attention to details have been lost from that generation to ours.
The engineers who designed the #Voyager probes half a century ago even thought of the possibility that a wrong sequence of commands may point the antenna dish away from earth (like someone did a couple of days ago).
And they implemented a self-adjusting mechanism that a few times a year scans the positions of a few known stars to infer the position of the earth, and point back the antenna in the right direction.
@blacklight And now look at the most recent Russian moon "landing". Perhaps they'd do better if they focussed more on space rocketry rather than the other type.
Do you remember when audio cassettes were seen as tools for piracy? When the industry used to drill in our brains the idea that if we were recording stuff from the radio to a cassette we were killing music as an art?
Guess what? Musicians actually loved the idea, they even released cassettes with a blank B side so you could record anything you liked on it.
And millions of kids who recorded their favourite songs or radio shows in the 1980s and early 1990s didn't kill the music.
On the contrary, the music industry itself ended up killing the music, by forcing artists to "play it safe" and repeat the same formulas again and again, so the cigar smoking capitalists wouldn't take too many risks on their investment into the new boyband.
"Don't do that, or you'll kill X" most likely won't kill X as a form of art, nor the artists. It'll just hurt those who have no interest in X as a form of art, who make profit out of somebody else's talent, and who want to have nobody pushing them to take new risk to modernize the industry.
For us, music is a form of art and communication. For them, it's just another mean of making money.
Do you remember when audio cassettes were seen as tools for piracy? When the industry used to drill in our brains the idea that if we were recording stuff from the radio to a cassette we were killing music as an art?
Guess what? Musicians actually loved the idea, they even released cassettes with a blank B side so you could record anything you liked on it.
@blacklight I also remember making dozens and dozens of mixtapes for friends who were interested in artists that weren't on the radio, and it isn't piracy if it turns out to be promotion.
Man... this hit me right in the soul. I'd never understood it quite like that before...spoken or written. Thank you...from my heart. A big twinge of nostalgia came with it too. Very well done. My hat's off to ya!
For me this is the last nail in the coffin for #Go.
I've never bought much into the language. I've been impressed by its constructs to natively manage and synchronize asynchronous operations, but its rigidity when it comes to programming paradigms (no proper object-oriented and functional constructs in the 21st century, seriously?) means that I see it as a language that seriously limits expressivity, and doomed to generate a lot of boilerplate. It's a language very good at solving the types of problem that are usually solved at Google (build and scale large services that process a lot of stuff in a way that the code looks the same for all the employees), and little more than that.
After #Rust really took off, I didn't see a single reason why someone would pick Go.
And now here we go with the last straw: Google has proposed to embed telemetry collection *into the language toolchain itself*. And, according to Google, it should be enabled by default (opt-out rather than opt-in), because, of course, if they make it an opt-in then not many people will explicitly enable a toggle that shares their source code and their usage of the compiler with one of today's biggest stalkers.
Not only, but Google went a bit further: "I believe that open-source software projects need to explore new telemetry designs that help developers get the information they need to work efficiently and effectively, without collecting invasive traces of detailed user activity".
No. Open-source doesn't need telemetry. Telemetry introduces brittle dependencies on external systems with no functional net gain, and that's at odds with the whole idea of building and running things on your own.
Open-source software has already a very well-established way of collecting feedback: open an issue on the project, and if you want to change something submit a PR. You don't need remote probes whose purpose is to funnel data back home. Even when done with the best intentions, that breaches the trust between the developer and the user - because data gets scooped out, and the systems that store and use that data aren't open. But, of course, if you've only used hammers in your life then the whole world will look like nails.
This could even backfire for Google. There are many applications out there where secrecy (and minimizing the amount of data that leaks outside of the network) is a strong requirement. These applications may start considering alternatives to a language that enables telemetry data back to an external private company by default.
If you build open-source projects in Go, it's time to drop it and start considering alternatives. The market for modern compiled language is much more competitive now than it was a decade ago. Many of us knew already that we couldn't trust a programming language developed by the largest surveillance company on the planet.
For me this is the last nail in the coffin for #Go.
I've never bought much into the language. I've been impressed by its constructs to natively manage and synchronize asynchronous operations, but its rigidity when it comes to programming paradigms (no proper object-oriented and functional constructs in the 21st century, seriously?) means that I see it as a language that seriously limits expressivity, and doomed to generate a lot of boilerplate. It's a language very good at solving the types of problem...
Well, I think we should give them the benefit of the doubt. After all, lest I remind you of these three (3) words for you from their credo:
"Do Evil"
Wait! Something's wrong there! I'm almost positive there used to be three (3) words there!
On another note, it is Apple that is the biggest surveillance company in the world. Yes, they exclude others at your expense being subject to their own dystopian intelligence gathering.
The only question I have after a shallow read of the #ATProtocol (the "specs" behind #Bluesky) is: why?
Why an entirely different protocol? Why not just an extension on top of #ActivityPub?
They wanted a bit more centralized control on top of a decentralized network - not a bunch of geeks running their own servers in the bedroom, but a "marketplace of companies", whatever that means, in charge of distribution and moderation? Fair enough, they are businesses that needs to make money after all.
But then why didn't they go for an extension that simply supports self-certifying data (signed using whatever centralized chain of certificates they want) on top of ActivityPub? The Fediverse and Twitter (or whatever they are going to call whatever they're trying to build now) could have immediately tapped into one another. They could have simply shown posts not certified by their chain as "unauthenticated", and we would have had almost 100% compatibility from day one.
Why a whole new protocol? Why build a new ActivityPub that does the same things as ActivityPub, but just with a layer of self-certification on top? Why call things differently and break compatibility, instead of extending the open standards?
It takes time and resources to build a new protocol, validate it, get developers to adopt it and build stuff on top of it, and add yet more layers of bridges and mappers to/from what exists already. And yes, I know that the alternative approach that I am promoting is exactly that "embrace+extend" that we all know how it ends up. And I'm also aware that are valid arguments to keep the streams separate. But I'm not sure if the best alternative is to let the rest of the world build their own brand of our open protocols (R&D these two balls: these guys are simply repackaging a decade of work on open standards and they are trying to profit from it), while we either keep digging our niches or play a catch-up game with bridges and mappers to the outside world.
The only question I have after a shallow read of the #ATProtocol (the "specs" behind #Bluesky) is: why?
Why an entirely different protocol? Why not just an extension on top of #ActivityPub?
They wanted a bit more centralized control on top of a decentralized network - not a bunch of geeks running their own servers in the bedroom, but a "marketplace of companies", whatever that means, in charge of distribution and moderation? Fair enough, they are businesses that needs to make money after all.
@blacklight After reading #ATProto yesterday, and despite it has a looooot of moving parts (some of which are still in its infancy), I think it adds some interesting concepts. For example:
- the use of decentralized IDs + a cryptographically secure mechanism to migrate instances,
- stronger schemas and guarantees for server-to-server communications,
- separation of "small world" and "big world" networking,
- separation of the "speech layer" from the "reach layer"
@blacklight dang. I was wondering the same thing when I read through that as of last night. I hope that an AP developer ends up joining the fray over there, because then there might be a chance that the AT Protocol will support AP, or at least be able to communicate with it.
@blacklight
I didn't read the spec, but from what I heard from first opinions, they want to make user own their identity. In ActivityPub your identity belongs to server you're using. So when you're migrating between the nodes, you're dependent on the will of both servers to say "yeah, that's still him".
Currently Fediverse is like using custodial Bitcoin wallets without having ability to store your keys yourself. You can move from one service to another, but you can't really be sure that your identity is yours.
And it's a problem I saw in the Fediverse long before I heard about BlueSky.
Don't misunderstand me. I'm not very sympathetic to what they're doing. But from what I heard (and I could've hear wrong) they solved one of most important AP problems.
@blacklight
I didn't read the spec, but from what I heard from first opinions, they want to make user own their identity. In ActivityPub your identity belongs to server you're using. So when you're migrating between the nodes, you're dependent on the will of both servers to say "yeah, that's still him".
Browsing projects on #Sourcehut reminds me of what FLOSS development looked like 15-20 years ago. Ugly interfaces that were just thin layers above the code, barely any README (let alone wikis, or any form of easily accessible and structured documentation), and let's not mention accessibility on mobile.
How are we supposed to build the foundations of tomorrow's FLOSS if we use tools that look even more outdated than Craigslist? How are we supposed to have any credibility when we tell people "stop using Github, try Sourcehut instead"? How do we expect to create user engagement? How do we expect somebody who's not a developer to use software that doesn't even come with an easily accessible documentation?
Browsing projects on #Sourcehut reminds me of what FLOSS development looked like 15-20 years ago. Ugly interfaces that were just thin layers above the code, barely any README (let alone wikis, or any form of easily accessible and structured documentation), and let's not mention accessibility on mobile.
@blacklight Impossible to decide where to start replying to this multi-branched thread (and, oh the irony, Mastodon doesn't do a good job at showing those branches), but 10000 % agreed with what you said.
A few folks monopolizing the replies essentially saying "#Sourcehut is just fine" illustrates the problem perfectly.
@blacklight Yeah, this is why I recommend/use @CodeBerg. No need to perpetuate FLOSS arrogance; it benefits no one (apart from the egos of some über geeks, I guess.)
When I see SourceHut it’s an immediate turn off for me for a project. It says “Keep Out, Über Geeks Only.”
(And this from someone who’s been making things with computers for over three decades. I just don’t like private members’ clubs even if I happen to meet their eligibility requirements.)
To future historians—not just of computing, but of humanity—the current period will be a dark age.
How was Facebook used by students in the 2010s? We cannot show you, that version of Facebook is not hosted anywhere.
How did MySpace look around 2009? We don't really know, the Wayback Machine only shows a limited amount of static content, and there may only be a few surviving screenshots
What correspondence did Vint Cerf have as president of the ACM with other luminaries of computing industry and research? We do not know; Google will not publish his emails.
What was it like playing Angry Birds on an iPhone 3G? We do not know; Apple is no longer distributing signed receipts for that binary.
What did the British cabinet discuss when they first learned of the Coronavirus pandemic? We do not know; they chatted on a private WhatsApp group.
What books were published analysing the aftermath of the Maidan coup in Ukraine? We do not know; we do not have the keys for the Digital Editions DRM.
How was the coup covered in televised news? We do not know; the broadcasters used RealVideo and Windows Media Encoder and we cannot read those files.
We have to ask ourselves how we are going to preserve and transmit knowledge about our age to the next generations. Knowledge about an age where information is produced, consumed and discarded within hours, days or months, or where it's only stored on the server rooms of a handful of corporations, with no guarantees that those businesses will exist in the future, and with no way of accessing that information unless a certain set of regulatory, hardware, software pre-conditions are met.
That's why projects like the Internet Archive deserve more recognition and funding. That's why web scraping should not only be a civic right, but a civic duty to the next generations. Otherwise all the knowledge about the great age of information will be transmitted orally - with all the distortions that such transmission implies.
To future historians—not just of computing, but of humanity—the current period will be a dark age.
How was Facebook used by students in the 2010s? We cannot show you, that version of Facebook is not hosted anywhere.
How did MySpace look around 2009? We don't really know, the Wayback Machine only shows a limited amount of static content, and there may only be a few surviving screenshots
Gregory's Server
