Hi, my (host)name is rpi. I’m a Raspberry Pi Zero W running NetBSD in read-only mode. Stefano has decided that I will serve as a mirror for the IT Notes blog, so various Varnish reverse proxies will also direct traffic to me - some as a primary server, others as a failover. I only respond to the reverse proxies via IPv6 and I’m connected to the USB output of a Mikrotik switch, so my existence depends on that switch. Even if I'm connected via wifi only. When it gets rebooted, I’ll disappear for a while too, but since I’m in read-only mode, I’ll come back just as I was before.
After 7 years sitting in a drawer, Stefano decided it was time for me to go operational. When Stefano updates the blog, the script that generates it will connect to me, remount me in read-write mode, copy the new files, and I’ll return to read-only.
Oh, by the way, there’s no external web server involved—I’m running bozohttpd, integrated into NetBSD’s base system, and started by inetd when needed.
Hi, my (host)name is rpi. I’m a Raspberry Pi Zero W running NetBSD in read-only mode. Stefano has decided that I will serve as a mirror for the IT Notes blog, so various Varnish reverse proxies will also direct traffic to me - some as a primary server, others as a failover. I only respond to the reverse proxies via IPv6 and I’m connected to the USB output of a Mikrotik switch, so my existence depends on that switch. Even if I'm connected via wifi only. When it gets rebooted,...
@stefano You have a distant relative living in India, who is catching dust just like you were. It's waiting, sitting idle trying to find its purpose in life...
I'm glad to announce the release of version 2.66 of #snac, the simple, minimalistic #ActivityPub instance server written in C. It includes the following changes:
As many users have asked for it, there is now an option to make the number of followed and following accounts public (still disabled by default). These are only the numbers; the lists themselves are never published.
Some fixes to blocked instances code (posts from them were sometimes shown).
Fixed non-appearing buttons Approve and Discard if the account requesting a follow were being followed.
I'm glad to announce the release of version 2.66 of #snac, the simple, minimalistic #ActivityPub instance server written in C. It includes the following changes:
As many users have asked for it, there is now an option to make the number of followed and following accounts public (still disabled by default). These are only the numbers; the lists themselves are never published.
Ok, here’s my first IT Horror Story of the Saturday night. It's a bit long.
I was at a client, a healthcare facility, to replace some hard drives. They didn’t want to spend money, so we had to keep the current setup running, which was outdated and unreliable.
Now I can say it: they didn’t want to spend because the general manager’s goal was to give work to a company of his friends, who were already providing support on two VMs, and he wanted to hand everything over to them. The IT manager hadn’t yet understood the financial interests of these people and still believed everything was in good faith, that they really didn’t have the funds. We were holding everything together with duct tape, but it was working and stable. I had set up a "cluster" with OpenNebula and GlusterFS for storage (later replaced with MooseFS and then with Ceph), using all available hardware.
We scheduled an intervention and notified everyone to disconnect and shut down the machines by 12:30. By 13, we had completed the backups, aiming to start the intervention by 14 and get the work up and running by 15:30. The goal was to update the systems and check the disks. We shut down all the VMs, had everyone disconnect. It was lunchtime.
We updated the servers, rebooted them. One of the disks started throwing errors. GlusterFS, for some reason I never really investigated (I have my theories, which I’ll share later, but from that day forward, GlusterFS no longer exists for me), decided to overwrite both that disk and its replica with zeros. I hadn’t changed anything.
Panic – there were backups, but on a USB 2 disk (old servers, no USB 3)! I immediately stopped everything. I was almost fainting. The IT manager didn’t understand what had happened, so I explained it to him. He announced to everyone that we would cancel the rest of the intervention, restore from backup, and have the work back up by 15:30 as planned, prioritizing the most critical VMs.
The "competing" company reached out. They had powered on one of their VMs and started "doing their own interventions." Even though they had been warned not to do anything. And they complained "we" had lost some of their data. Of course, the manager and those guys went "carpe diem": they put me under accusation, saying I had undoubtedly made a mistake that led to "lost data." I wrote a technical report explaining what I had seen, noting various SSH logins from those guys during the intervention. The "history" had been erased. Of course, not by me.
They continued to harass me for a while. The last thing they asked was for me to go to a meeting "to explain in person." They tried to schedule it the day before my wedding. And they knew it.
They threatened to ask me for an unspecified (high) financial compensation for 'the lost data.' What lost data? The ones that, allegedly, the other company would have entered in the meantime.
Final result: no problem for me (I hadn’t done anything wrong), the backups were fine, they only calmed down when I proved (logs in hand) that I wasn’t the only one connected to that machine, and my witnesses (two colleagues and the IT manager) had seen all my actions, confirming I hadn’t done anything wrong.
In the end: I realized they would do it again and I left the client – even the IT manager decided to resign and change jobs. The general manager managed to install, at astronomical figures, the company he wanted to place. After two months, they got their hands on the system and broke it. They asked me for assistance, which I refused. At any price.
After a few years, I found out that the general manager ended up in jail for corruption, bribes, and for favoring his friend companies in many sectors.
Ok, here’s my first IT Horror Story of the Saturday night. It's a bit long.
I was at a client, a healthcare facility, to replace some hard drives. They didn’t want to spend money, so we had to keep the current setup running, which was outdated and unreliable.
Now I can say it: they didn’t want to spend because the general manager’s goal was to give work to a company of his friends, who were already providing support on two VMs, and he wanted to hand everything over to them. The IT manager hadn’t...
@stefano i could react “ unbelievable! ”, but the sad thing is that I certainly DO believe such an horror story. I guess a lot of us have already seen this kind of very shameful practices. I did.
This window was my view from the room where I worked in the 'old' house, from 2008 to 2015.
Many of the projects I have up and running today were conceived, studied, implemented, and understood from that room, gazing out of that window. It offered me peace, relaxation, and focus, with its bucolic, green backdrop, connecting me with nature and a slower pace of life.
This window was my view from the room where I worked in the 'old' house, from 2008 to 2015.
Many of the projects I have up and running today were conceived, studied, implemented, and understood from that room, gazing out of that window. It offered me peace, relaxation, and focus, with its bucolic, green backdrop, connecting me with nature and a slower pace of life.
I really struggle to understand the direction of today's society.
For years, we've been saying we need to reduce emissions and consumption, removing chargers from smartphone boxes "to pollute less," then the "AI" comes along and we reopen fossil fuel power plants, increasing consumption and emissions for... well, who knows why!
We make people feel guilty for not switching to an electric car (or one with high energy efficiency), yet we fly for pure leisure, just to get "a few more likes" on YouTube.
We create increasingly efficient electronic devices, focus on the consumption and emissions of data centers (local or remote), and then stop optimizing code and dependencies because "there’s autoscaling" and "resources are cheap."
I really struggle to understand the direction of today's society.
For years, we've been saying we need to reduce emissions and consumption, removing chargers from smartphone boxes "to pollute less," then the "AI" comes along and we reopen fossil fuel power plants, increasing consumption and emissions for... well, who knows why!
@stefano that's why #righttorepair
and Open Source is very important imho
A computer is supposed to be compliant for 36 months now where i work.
Insane, i could still rock my Powerbook G3
If i could understand computer software more.
M$ Teams take more resources then that machine has 😆😆
In recent days, I've received a few messages and noticed some posts regarding my articles on the blog. The criticism is mainly about the fact that, in some cases, I don't document "every" step but assume a basic understanding of the topics discussed. For example, if the article is about "how to install Y within a FreeBSD jail," I don't document how to install FreeBSD, what jails are, or how they should be managed, etc. In some cases, I refer to previous articles, but my aim is never to create "for dummies" tutorials. I believe that self-hosting, if done without awareness, creates more problems than it solves.
I’ll probably need to publish an article specifically about this—and maybe link it to a menu at the top of the page to explain it. My approach has usually been to provide tools to understand how I solved a problem, not to hand out "ready-made solutions"—the goal is to help people understand, not to mindlessly copy without comprehension.
After all, the blog is called "IT Notes," and they are my notes, turned into articles, mostly related to direct experiences I’ve just had. It’s not called "IT Course"—those, when necessary, I create in other ways.
In recent days, I've received a few messages and noticed some posts regarding my articles on the blog. The criticism is mainly about the fact that, in some cases, I don't document "every" step but assume a basic understanding of the topics discussed. For example, if the article is about "how to install Y within a FreeBSD jail," I don't document how to install FreeBSD, what jails are, or how they should be managed, etc. In some cases, I refer to previous articles, but my aim is never to create "for...
"If you wish to make an apple pie from scratch, you must first invent the universe." ー Carl Sagan.
IMHO, it's totally OK to post apple pie recipes, without telling users how to sync /usr/src && make universe though! ;) (Or even make buildworld && make installworld for that matter, instructions for such things are definitely available upstream.)
@best_of_mltshp This is what I’ve been trying to explain— this mysterious art deco city— but also the crucible of labor revolts— and that undercurrent of the appalachian countryside— it’s an intersection of worlds. worlds that invented each other—
This morning I noticed a problem. The Mastodon instance actually has two Jail environments, one of which is on the VPS of the reverse proxy. Two days ago, I stopped it (it's not necessary, but serves as support and failover if the primary one is down), and it remained stopped.
Push notifications via Google FCM stopped arriving, while those via UnifiedPush were still coming through.
After a morning of debugging, I figured it out: the Jail environments on the bigger BSD Cafe VPS route all traffic via Wireguard, and there was an MTU issue.
The push notifications via FCM contain the full notification data in an encrypted form, while those via UnifiedPush simply "wake up" the app, which then fetches the notification.
In other words, the first type had large packets that were being blocked due to the MTU discrepancy.
I suspect this might have affected the functioning of the other Jails as well, so I modified all of them, and now everything should be fine.
This morning I noticed a problem. The Mastodon instance actually has two Jail environments, one of which is on the VPS of the reverse proxy. Two days ago, I stopped it (it's not necessary, but serves as support and failover if the primary one is down), and it remained stopped.
Push notifications via Google FCM stopped arriving, while those via UnifiedPush were still coming through.
After a morning of debugging, I figured it out: the Jail environments on the bigger BSD Cafe VPS route all traffic...
Gregory's Server
