Email or username:

Password:

Forgot your password?
32 posts total
Lennart Poettering

Here's a blog story with links and very brief summaries of all those #systemd257 stories on Mastodon:

0pointer.net/blog/announcing-s

Enjoy! And stay tuned for #systemd258!

Lennart Poettering

3️⃣7️⃣ Here's the 37th post highlighting key new features of the current v257 release of systemd. #systemd257

In systemd v256 we added a small tool "systemd-ssh-proxy" whose job is to allow connecting to local VMs with ssh via the AF_VSOCK protocol (as opposed to AF_INET/AF_INET6). It acts as host-side counterpart to the guest-side systemd-ssh-generator that automatically binds sshd to AF_VSOCK.

In systemd v257 the functionality has been updated so that instead of specifying…

Lennart Poettering

…the AF_VSOCK "CID" (which is like an IP address, i.e. an identifier for the local VM) you can specify a friendly machine name, if the VM in question is registered with systemd-machined. systemd-vmspawn sets things up that way out of the box, of course. That means, with current off-the-shelf systemd inside a VM and on the host you can now just do "ssh machine/foobar" to connect to a local VM called "foobar", via AF_VSOCK, i.e. independently of any fragile network.

Lennart Poettering

3️⃣2️⃣ Here's the 32nd post highlighting key new features of the current v257 release of systemd. #systemd257

One of the features we added on early to systemd was coredump processing. We wanted that crashing processes on one hand could be treated very much like any loggable event, and on the other hand be truly and immediately useful, i.e. the log messages generated should already carry a fully symbolized backtrace.

The path towards that goal was rocky, but today I think we…

Lennart Poettering

…are in a pretty good position: we use libdw to generate the stack trace, running inside a local sandbox (since generating stacktraces means analyzing frequently corrupted, possibly differently privileged, complex data, which is hence security sensitive par excellence), and since the relevant distributions now ship minidebuginfo packages and are built with frame pointers enabled the default stacktraces you get this way are typically quite useful – without having to bother with gdb or so.

Lennart Poettering

3️⃣0️⃣ Here's the 30th post highlighting key new features of the current v257 release of systemd. #systemd257

Yesterday we talked about "systemd-analyze" already, and the newly added support for decoding process capability masks.

With v257 of systemd, there's another new addition to the systemd-analyze tool: "systemd-analyze smbios11". This new command will look for your system's SMBIOS table, specifically object type 11, and show its contents.

Anselm Schüler

@pid_eins I really like your posts about this. Given the sheer number of highlights this time, though, would you consider collecting these in a more practical format somewhere? It can be a pain to scroll through Mastodon for this, and it presents the information much better than release notes.

Lennart Poettering

2️⃣8️⃣ Here's the 28th post highlighting key new features of the current v257 release of systemd. #systemd257

(systemd v257 has been released now, but I am not done with this series yet, 10 more posts to go, including this one.)

Early during systemd's creation we introduced the /etc/os-release file which carries information about the system's OS in a generic fashion. It has found very wide adoption, even beyond the systemd world (for example Solaris and FreeBSD have it).

Lennart Poettering

I guess many of the systemd detractors don't even realize that /etc/os-release is a systemd-ism.

Over the years the failed acquired various fields. It no longer just describes the distribution a system was installed with, but on image based systems it describes the image itself. It gained fields for describing the end of support, and where to find the release notes. And it's extensible: various distributions add their own fileds (hopefully properly prefixed by a namespace prefix!).

Lennart Poettering

2️⃣7️⃣ Here's the 27th post highlighting key new features of the upcoming v257 release of systemd. #systemd257

Since a longer time systemd's service management supports various sandboxing options: concepts such as PrivateMounts=, PrivateNetwork=, PrivateDevices=, ProtectSystem=, ProtectHome=, PrivateUsers= and so on, all lock down what a service can see and do, usually by means of Linux process namespacing.

With v257 we add one more knob to this: PrivatePIDs=. This wraps Linux PID namespacing.

Lennart Poettering

Or in other words: when enabled the service will run as PID 1 in its own process namespace.

Up to this point we avoided exposing PID namespacing as part of service namespacing, following the thinking that service management should make stuff invisible and inaccessible, but not open "a new world", but unlike the other namespacing types PID namespacing is really about opening "a new world", with its own PID 1 and so on.

Or in even other words: we thought that PID namespacing is a concept…

Lennart Poettering

2️⃣6️⃣ Here's the 26th post highlighting key new features of the upcoming v257 release of systemd. #systemd257

In particular on image based systems that receive comprehensive updates regularly it makes sense to automatically reboot them at the right times, so that the newer kernel and new userspace code definitely gets used, and is more than just dead weight on disk.

When multiple components of the OS (let's say kernel on one hand and the root fs on another) are updated, it's also essential…

Lennart Poettering

…that the system is rebooted as often as really necessary but not more than that, in order to maximize uptime, and not annoy the user needlessly. After all, rebooting means disruption, and unavailability of the system's services.

With systemd v257 the systemd-logind service gained a new centralized knob for designating a specific "maintenance time" for systems via the new DesignatedMaintenanceTime= setting in logind.conf.

It takes a calendar time expression (in the same format…

Lennart Poettering

2️⃣3️⃣ Here's the 23rd post highlighting key new features of the upcoming v257 release of systemd. #systemd257

In systemd, as mentioned in one of the previous installments, we are adopting the Varlink IPC at more and more places.

To interface with Varlink IPC from the command line we provide the "varlinkctl" tool. "varlinkctl introspect <socket>" for example introspects which method calls, types, and errors a service provides. Similar, "varlinkctl call <socket> <method> <json>" calls a method.

Lennart Poettering

Here are two examples using that:

varlinkctl introspect /run/systemd/io.systemd.Credentials

and

varlinkctl call /run/systemd/io.systemd.Credentials io.systemd.Credentials.Encrypt '{"text":"foobar"}'

The 2nd argument always specifies the entrypoint socket to talk to. In most cases where you call this locally that's the file system path of an AF_UNIX socket. However this can also be the path to an executable, in which case the executable is invoked and told via $LISTEN_FDNAMES (i.e.

Mickaël Salaün

@pid_eins can we pass file descriptors to local services?

Lennart Poettering

2️⃣0️⃣ Here's the 20th post highlighting key new features of the upcoming v257 release of systemd. #systemd257

systemd-journald is systemd's log service that collects logs from system services, acquires some metadata and then stores all that in a mostly append-only linear database on disk, keyed by a multitude of metadata fields.

One such field is the "invocation ID". The invocation ID is a 128bit randomized ID that every systemd unit gets freshly assigned whenever it enters its start phase.

Lennart Poettering

Thus, if the same service gets started 5 times iteratively, it will have 5 different invocation IDs assigned. And if you want to pinpoint a specific invocation of a service, it's the invocation ID you can use to uniquely identify it. (Besides the use as metadata when logging the invocation ID is also passed via the env var $INVOCATION_ID to the services btw; and systemctl status also shows it these days.)

One could say that what the Linux boot ID does for the system…

Lennart Poettering

8️⃣ Here's the 8th post highlighting key new features of the upcoming v257 release of systemd.

A longer time ago systemd introduced JSON based user records as an extension of classic UNIX `struct passwd`. These records can be provided via Varlink IPC or via drop-in files. The much richer set of account settings is documented here:

systemd.io/USER_RECORD/

Lennart Poettering

These fields are consumed and acted on by systemd-logind, systemd-homed and other components of systemd.

One field is particularly interesting: it contains a field for the public SSH keys of an account (i.e. the stuff you traditionally find in ~/.ssh/authorized_keys). By making it part of the user record itself it can be accessed and used by SSH without access to the home directory of the user.

Lennart Poettering

7️⃣ Here's the 7th post highlighting key new features of the upcoming v257 release of systemd. #systemd257

The graphical login prompt you see when your computer boots up is a sensitive UI: typically, when starting to work, without much thinking you'll type in your username and password, expecting it to log you in and provide you with your desktop session. However, what if someone just opened a website in a browser in full screen mode with contents that just *looks* like your login screen, …

Lennart Poettering

… but actually is just some malware that exfiltrates the password you type in?

Since this kind of attack scenario is not new, many OSes provide a "SAK" concept, which stands for "Special Attention Key". The idea is that there's a special key combination you can hit first, which no web page, or web browser, or app, or even desktop environment could possibly hook into that always brings you back to your *real* login screen, regardless where you are.

Lennart Poettering

5️⃣ Here's the 5th installment of posts highlighting key new features of the upcoming v257 release of systemd. #systemd257

Since its beginnings systemd has been a heavy user of the D-Bus IPC system. It provides D-Bus APIs, it calls D-Bus APIs, it schedules activation of the D-Bus broker, and even provides its own C D-Bus client library (sd-bus).

However, since early on our use of D-Bus was not without various major problems. One of the biggest goes something like this:

Lennart Poettering

D-Bus' model is built around a central broker daemon, which is started during boot, but unfortunately relatively late (i.e. together with other, regular daemons instead of early boot or the initrd). However, systemd brings up the system as a whole and hence needs IPC from earliest moment on.

And then there are various components of systemd that the D-Bus broker relies on (i.e. consumes functionality of) and hence cannot themselves provide their services on D-Bus, …

Lennart Poettering

4️⃣ Here's the 4th post highlighting key new features of the upcoming v257 release of systemd. #systemd257

One of the key features of systemd we have talked about in the past years are UKIs, i.e. "unified kernel images", which is a combination of a Linux kernel, an initrd, and more into a single unified PE binary, that can be signed as a whole for SecureBoot, measured as a whole and updated as a whole.

In my PoV UKIs are a central concept of securing the Linux boot process.

Lennart Poettering

But: they do have some disadvantages. They typically imply (not strictly, but typically) that they are built on OS vendor build systems instead of locally. This is different from the status quo ante, where the initrd is typically built on the deployed system (at least on generic distros), and thus highly adapted to the local system.

UKIs being vendor-built hence means they are a lot more rigid, less flexible than the traditional way. So far this meant you'd have to settle…

Julian Andres Klode 🏳️‍🌈

@pid_eins UKIs are fine, but systemd-stub churn every release makes it more of a liability than a stable foundation to build on.

Stuff like sysexts etc could poke holes in established security models similar (but restricted in scope) to how systemd poked giant holes into full disk encryption when it added DDI automount.

It's a massive effort to keep up with changes and validate them and it's pretty concerning. So you might end up not doing that, and end up with vulnerabilities in your product.

Lennart Poettering

In systemd we started to do more and more Varlink IPC (instead of or 9n addition to D-Bus), and you might wonder what that is all about. In this AllSystemsGo talk I try to explain things a bit, enjoy: media.ccc.de/v/all-systems-go-

Chris Friedt

@pid_eins - Varlink reminds me of the IPC system we used at BlackBerry like 12 years ago. I guess the big difference was back then it was basically JSON with additional binary integer types. Not exactly like CBOR.

Lennart Poettering

So, if you ask me what my takeaway from the Crowdstrike issue is, I'd say: boot counting/boot assessment/automatic fallback should really be a MUST for today's systems. *Before* you invoke your first kernel you need have tracking of boot attempts and a logic for falling back to older versions automatically. It's a major shortcoming that this is not default behaviour of today's distros, in particular commercial ones.

Of course systemd has supported this for a long time:

systemd.io/AUTOMATIC_BOOT_ASSE

So, if you ask me what my takeaway from the Crowdstrike issue is, I'd say: boot counting/boot assessment/automatic fallback should really be a MUST for today's systems. *Before* you invoke your first kernel you need have tracking of boot attempts and a logic for falling back to older versions automatically. It's a major shortcoming that this is not default behaviour of today's distros, in particular commercial ones.

Show previous comments
Bou

@pid_eins wait, distros could just just enable it and they don't? How come?

Lars Marowsky-Brée 😷

@pid_eins The shocking thing is that this was a requirement for Carrier Grade Linux two decades ago already.
When it comes to reliability and availability as part of dependable computing, our (distributed or not) systems have somewhat regressed as they were scaled up.

Anthk

@pid_eins

You mean, like keeping old grub/lilo entries and kernels since forever?

Lennart Poettering

As I understand the BSOD today became more popular than ever, truly becoming mainstream and reported about all over the news. Of course, in systemd we are ahead of the curve, as usual, and if you too want to experience your very own BSOD we have your back. Enjoy:

freedesktop.org/software/syste

Finally no need to feel left out again, just because you use Linux! 💖💘💝

Mike Yuan

@pid_eins (And do note that kernel is following suit with the new DRM panic stuff ;-)

ck

@pid_eins So next time a BSOD goes round the world on a low res screen, Microsoft will finally plausible deniability, thus completing you evil plan 😈

Lennart Poettering

1️⃣6️⃣ Here's the 16th installment of posts highlighting key new features of the upcoming v256 release of systemd.

(Sorry for dropping the ball on posting these for a while!)

The last feature I want to discuss in this series of postings is the new "capsule" concept of systemd v256.

systemd can be invoked in two contexts: as a system manager, i.e. PID 1, where it manages, well, the system. And as a user manager where it runs services for a specific user.

Lennart Poettering

At any time on a single systemd machine you'll have 1 system manager and 0…n user managers running. This model is built around the traditional UNIX security model: UIDs are the primary security concept of the system, and thus we give each relevant UID a service scope of its own.

While the original purpose of the per-user service manager is to provide functionality for actual human users logging in interactively people have been using the concept for other purposes as well:

Lennart Poettering

1️⃣3️⃣ Here's the 13th installment of posts highlighting key new features of the upcoming v256 release of systemd.

ssh is widely established as *the* mechanism for controlling Linux systems remotely, both interactively and with automated tools. It not only provides means for secure authentication and communication for a tty/shell, but also does this for file transfers (sftp), and IPC communication (D-Bus or Varlink).

Lennart Poettering

It relies on TCP as network transport, which is great for remote operation around the globe but really sucks for local communication with a VM and similar, as it usually requires delegation of an address space, dhcp lease, dns and so on, which while manageable are certainly a major source of mistakes, fragility and headaches. In particular it means that logging into a system to debug networking doesnt really work since without working networking you cant even log in. Sad!

Lennart Poettering

7️⃣ Here's the 7th installment of my series of posts highlighting key new features of the upcoming v256 release of systemd.

In systemd we put a lot of focus on operating with disk images, specifically file system images that carry an expressive GPT partition table – something that we call DDIs ("Discoverable Disk Images").

Lennart Poettering

DDIs are supposed to carry dm-verity authentication information, i.e. every single access to them is typically cryptographically protected, and linked back to a set of signing keys maintained by the system (ideally in the kernel keyring). systemd uses DDIs for the system itself, for systemd-nspawn containers, for systemd portable services, for systemd-sysext system extensions, for systemd-confext configuration extensions and more.

Lennart Poettering

5️⃣ Here's the 5th installment of my series of posts highlighting key new features of the upcoming v256 release of systemd.

I am pretty sure all of you are well aware of the venerable "sudo" tool that is a key component of most Linux distributions since a long time. At the surface it's a tool that allows an unprivileged user to acquire privileges temporarily, from within their existing login sessions, for just one command, or maybe for a subshell.

"sudo" is very very useful, as it…

Show previous comments
Sebastian Wick

@pid_eins suid programs executing in the environment of the parent process means that I might become root in a user namespace and get the filesystem view of the current mount ns. This won't work with your approach, will it?

Matthias Klumpp

@pid_eins This is very nice - especially having it provide a clean context, that saves a ton of headaches and sanitization I need to do when using "sudo" in other programs! (not like that's an amazing thing anyway, but occasionally it's useful and justified)
Only having run0 coloring / changing the output of the called command is not something I always like, but maybe that can be optionally disabled...

Go Up