Gregory's ServerI just streamed for an hour from this and nothing crashed??
|
80 comments
 Today in Wayland compositor profiling! Turns out closing a shm pool file descriptor can result in a fat stall of up to like 6 ms with the kernel waiting on some spinlocks. Which is extra fun when you realize it covers the entire frame budget of your 165 Hz screen, and some clients are sometimes doing it every frame! I'm trying a "dropping thread" workaround where the fd closing happens on a separate thread. Appears to work at the first glance. new main loop stall dropped and it is, uhhhhh, epoll_wait doing blocking disk decryption for solid 8 ms? is that a thing that it does? seems to have happened once over a long period but still Found the same disc decryption during rendering. Does it just randomly decide to do it or something? Aside from this and some other weirdness, not a single dropped frame on my slower laptop! (which is admittedly just 60 Hz) Thought of another thing to plot in Tracy: target presentation time offset! This is the difference between when a frame was shown on screen and the target time that we were rendering for. Here you can see data across 17 seconds of runtime while recording with OBS. Offset on both monitors fluctuates within a few microseconds around zero, which means that our rendering lands right on time. It's also common to see one frame worth of offset like on this zoomed-out screenshot. This happens when the compositor wakes up from idling too late into the monitor refresh cycle and doesn't manage to render a new frame in time. I'm still working on niri btw (and using it myself too). Today I finally finished a window layout refactor that was due from very early on. Now the layout always works correctly, with all the paddings, struts, fullscreen windows and animations. It's tricky because while most of the logic operates only on the "working area" (view excluding struts), fullscreen windows in particular must cover the entire view area, while otherwise acting as just another regular window column. niri development is ongoing, getting a lot of help from kchibisov too. Today I implemented an interactive area screenshot capture tool. Almost like a mini screenshot UI :ablobcatbongo: The development pace had slowed down a little, so I've tagged an alpha release for niri: https://github.com/YaLTeR/niri/releases/tag/v0.1.0-alpha.1 Also made a COPR for the occasion, if you're on Fedora: https://copr.fedorainfracloud.org/coprs/yalter/niri/ A month has passed and a number of important additions have landed in niri, so here's a second alpha release: https://github.com/YaLTeR/niri/releases/tag/v0.1.0-alpha.2 Highlights include relative-pointer and pointer-constraints which let Xwayland masterfully handle 3D games mouse look, and popup unconstraining which prevents popups from opening off-screen. I actually made popups place within their window with some padding, which looks quite nice. Aaahhhhhh another difficult refactor down and niri now does multi-GPU! By which I mean that monitors plugged into secondary GPUs will now light up and work. All the screenshot UI and screencast portal stuff also works just fine. Wouldn't be able to do this as quickly without Smithay's MultiRenderer support and lots of help from @drakulix 😄 I went for the easier strat of always rendering on the primary GPU, but you can also pick render GPU dynamically, which apparently cosmic-comp does, cool! Tagged niri v0.1.0-alpha.3 with multi-GPU support, borders and other improvements! Multi-GPU was one of the bigger things I wanted to get done before going out of alpha so I guess I'm slowly getting there. Turns out that if you implement xdg-decoration in your compositor but tell clients that you want CSD, then SDL2 + libdecor clients will break due to a bug. The bug is already fixed, but the fix hasn't made it to any SDL2 release yet, let alone all the runtimes and vendored copies. Hiding xdg-decoration from clients it is then I'm pretty excited to finally be able to "release" niri soon but god I'm glad I set aside a beta week for bugfixes. We already stumbled upon and fixed several issues A few latency tests before release confirm that niri's still doing good (at least on idle; I don't have any repaint scheduling yet but on idle it doesn't matter). The compositors are pretty much within the noise threshold from each other. Except some sway fullscreen bug and Shell losing one frame somewhere. Well, I'm happy to release the first stable version of niri, my scrollable-tiling compositor: https://github.com/YaLTeR/niri/releases/tag/v0.1.0 Very satisfied with the current state, even though there's plenty left to do. Took a lot of time and work but I've certainly learned a lot, and I'm glad to have contributed a bit to Smithay too! Before adding more animations into niri, I'm making a "visual tests" application. It shows a set of hardcoded scenarios which I can quickly go through and visually check that everything looks right. It uses solid color rectangles as "windows", but otherwise this is the real niri layout code and real niri + Smithay rendering code, drawing to a GTK GL area. For example, on the last test you can see that my offscreen code currently clips CSD shadows (during the open animation). visual tests are such a lifesaver, so glad I stole that idea from osu!lazer The window opening animation is now live as part of niri v0.1.2: https://github.com/YaLTeR/niri/releases/tag/v0.1.2 I'm really looking forward to more animations, but wow they sure do need a lot of care to get right in all the edge cases. Also, I added a way to programmatically invoke compositor actions, and turns out that's quite useful for making video demos! For a bit of fun I added gradient borders to niri. One of the visual tests for it turned out a bit mesmerizing to look at Tagged niri v0.1.3: https://github.com/YaLTeR/niri/releases/tag/v0.1.3 This one has much improved touchpad gestures with inertia, springs, rubberbanding and everything else I copied from libadwaita, my primary source for things that feel good :blobmiou: Also thanks @alice for helping and giving feedback on the gestures and for giving a try to the touch support! Just implemented something I've had in mind for a while: compositor-side blocking out of windows from screencasts! The compositor is the perfect place to do this since it can replace the window contents in the render tree, which will work fine with any kind of overlapping, transparency, etc. AND it will work with anything that records the screen through the portal, be it OBS or video meeting, or whatever. There's actually an important edge case here: if you open the screenshot UI while recording the monitor, then the screenshot UI preview will show the window, and OBS, recording the screenshot UI preview, will hence also show the window. There are trade-offs here for how you want this to work; for now I put a big warning around the option, and added a stricter mode that blocks out the window from ANY screen capture (which means you can't screenshot it). I think for the built-in screenshot UI this is solvable with one more layer of indirection (render the screenshot UI preview itself twice, once for screencasts, and once for the monitor). However, for third-party screenshot annotation tools, this will still be a problem. Implemented this idea. It means rendering each monitor 3 times always for the screenshot but maybe it's fine? On this laptop 3 monitors × 3 renders takes 2 ms, and there's some unnecessary blocking I forgot to remove. On the video, note how for me the screenshot UI has Secrets visible, but on the recording afterwards it's always blocked out. I may be having too much fun recording a demo for the release notes Took the whole yesterday and a bit of today, but I've got window closing animations working! These turned out to be tricky because they need storing a snapshot of the surface render tree to draw once the app is gone. Some apps may start destroying their subsurfaces before the main surface, like alacritty with its sctk CSD, making it very easy to miss parts of the window in the snapshot, and therefore in the closing animation. Also, windows closing to the left no longer shift the view! Definitely one of the most complex animations yet: window resizing. Just the crossfade effect itself took a while to get working with all the window geometries and buffer offsets, and then there's the whole multiple window orchestration with Wayland's asynchronous nature. (I don't do animation transactions yet, that'll be a whole other level of complexity on top.) Happy with the result though, and it's cool that it seamlessly works with block-out-from screencast. Window movement across columns is now animated too! These weren't complex per se, but very *finicky*. Spent quite a bit of time chasing down all the offsets and coordinates to add and subtract to avoid jumps, but it seems to all work well now! All the animations, plus VRR, today in niri 0.1.5: https://github.com/YaLTeR/niri/releases/tag/v0.1.5 I also remade the demo video to showcase the animations and some of the newer features! We've reached 1000 commits 😅 with an a bit of an anticlimactic one though Another tricky feature, rounded corners! Took several days, but I believe I've got a pretty complete implementation. You (manually) set the window corner radius and whether to force-clip the window. You can set radius per-corner to match GTK 3 apps. It works correctly with subsurfaces, blocked-out windows, transparency, gradient borders, resize and other animations. Optimization-wise, opaque regions and even overlay plane unredirection work where possible! Also, we've now got a little #niri setup showcase thread :blobcat: https://github.com/YaLTeR/niri/discussions/325 Added (stole from GNOME Shell as usual) a screen transition action, so now I can finally switch between dark and light in style (of course, it works with blocked-out windows) Okay, time for an actually useful feature: interactive mouse resizing (yes, finally). This was, as it goes, quite fiddly to implement, especially since niri has to negotiate with the window during the process. I also added a double-resize-click (i.e. trigger a resize twice quickly) gesture to reset the window height or to toggle full width. Suggested by FreeFull on our Matrix and worked out very well! Really starting to feel quite nice with mouse. (still no transactions yet) nvim really taking its time processing all this 1000 Hz worth of resizing lol Since I'm in a mouse gesture mood today: hooked up the horizontal touchpad swipe to Mod + middle mouse drag and omg it feels so good with the spring deceleration and all (of course it also correctly avoids the touchpad scaling, so that when using the mouse gesture, the cursor location is always exactly anchored to the view position) Now for something fun. I'm experimenting with the ability to set custom shaders for animations. Today I added custom shader support for window-close, which lets me make this cool falling down animation! This is entirely optional of course, and there's no performance impact if you don't use it. Also, custom shaders, like the rest of the niri config, are live-reloaded, making it easy to play around with them. Been fixing quite a bit of interactive resize jank and other small stuff since the last time, but also added custom shader support for window-open, thus completing it for all three main window animations (open, close, resize). Now I didn't actually have any good idea of what I might want in a window open custom shader (I like the default), so I made a simple expanding circle animation to showcase it. Niri 0.1.6 with interactive window resizing, rounded corners, named workspaces, mouse view scrolling, animation custom shaders, screen transition! https://github.com/YaLTeR/niri/releases/tag/v0.1.6 Didn't realize quite how many release notes there would be this time; even had to use an extra level of headings. 😅 Over the past few weeks I've been working on fractional scaling for niri. A simple implementation took about a day, but to do it *properly* I had to refactor the entire layout code to work in floating-point. The result is well worth it though. Borders, gaps and windows are always physical-pixel aligned, and not restricted to integer logical pixel positions. There's no blur or position-dependent +-1 px jank. Fractional-scale-aware clients remain crisp at any scale. So it turns out that changing PipeWire screencast stream resolution on the fly is actually not that hard! Which is great news because it's required (or at least very desirable) for implementing window screencasting. Phew, finished the initial implementation of window screencasting in niri! Complete with stream resolution change on window resizing. Some details are still iffy regarding frame timing and frame callbacks, especially to obscured windows, but it *should* work decently fine for now. Icons are missing in the portal dialog because apparently Shell keeps track of Wayland app ID to .desktop file mapping internally and returns the .desktop file name to the portal for it to get the icon. Which is a bit too much effort for me to replicate for now. 😅 Fixed transparency support. Turns out the BGRA format should've been in a separate pod, rather than as a choice in the same pod. Should've looked at Mutter code sooner as usual.. niri v0.1.7 is out with fractional scaling, window screencasts and many smaller improvements! |
Dmabuf screencasting is crazy good. Here's a histogram of the screencasting overhead on my 2560×1600@165 screen—the median is 300 microseconds, and the worst across 12,669 frames was just below 1 ms. Most of that time is spent rendering the frame, perhaps something could even be further optimized in Smithay.
And yeah, if you look at the profiling timeline, I zoomed it in such a way that almost the entire width is taken by one frame, that is 6.05 ms long. Most of it is completely empty!