It's 6 PM, and I just got my breakfast ¯\_(ツ)_/¯

One of our drives died today, but in a way that it blocked the boot process of the machine. So, I was waiting for the fix, which happened in a good enough time in the end.

Then zfs import -f zroot && zfs replace..., and we back to life like nothing happened.

We have daily database backups to another geographic location and all attachments are stored in S3. Even in case of total destruction of the data center, we can quickly restore the machine with exactly the same configuration thanks to NixOS, import database, and keep going.

So, nothing can happen to us except the short downtime.