Gregory's ServerJust implemented something I've had in mind for a while: compositor-side blocking out of windows from screencasts! The compositor is the perfect place to do this since it can replace the window contents in the render tree, which will work fine with any kind of overlapping, transparency, etc. AND it will work with anything that records the screen through the portal, be it OBS or video meeting, or whatever.
|
76 comments
 I think for the built-in screenshot UI this is solvable with one more layer of indirection (render the screenshot UI preview itself twice, once for screencasts, and once for the monitor). However, for third-party screenshot annotation tools, this will still be a problem. Implemented this idea. It means rendering each monitor 3 times always for the screenshot but maybe it's fine? On this laptop 3 monitors × 3 renders takes 2 ms, and there's some unnecessary blocking I forgot to remove. On the video, note how for me the screenshot UI has Secrets visible, but on the recording afterwards it's always blocked out. I may be having too much fun recording a demo for the release notes Took the whole yesterday and a bit of today, but I've got window closing animations working! These turned out to be tricky because they need storing a snapshot of the surface render tree to draw once the app is gone. Some apps may start destroying their subsurfaces before the main surface, like alacritty with its sctk CSD, making it very easy to miss parts of the window in the snapshot, and therefore in the closing animation. Also, windows closing to the left no longer shift the view! Definitely one of the most complex animations yet: window resizing. Just the crossfade effect itself took a while to get working with all the window geometries and buffer offsets, and then there's the whole multiple window orchestration with Wayland's asynchronous nature. (I don't do animation transactions yet, that'll be a whole other level of complexity on top.) Happy with the result though, and it's cool that it seamlessly works with block-out-from screencast. Window movement across columns is now animated too! These weren't complex per se, but very *finicky*. Spent quite a bit of time chasing down all the offsets and coordinates to add and subtract to avoid jumps, but it seems to all work well now! All the animations, plus VRR, today in niri 0.1.5: https://github.com/YaLTeR/niri/releases/tag/v0.1.5 I also remade the demo video to showcase the animations and some of the newer features! We've reached 1000 commits 😅 with an a bit of an anticlimactic one though Another tricky feature, rounded corners! Took several days, but I believe I've got a pretty complete implementation. You (manually) set the window corner radius and whether to force-clip the window. You can set radius per-corner to match GTK 3 apps. It works correctly with subsurfaces, blocked-out windows, transparency, gradient borders, resize and other animations. Optimization-wise, opaque regions and even overlay plane unredirection work where possible! Also, we've now got a little #niri setup showcase thread :blobcat: https://github.com/YaLTeR/niri/discussions/325 Added (stole from GNOME Shell as usual) a screen transition action, so now I can finally switch between dark and light in style (of course, it works with blocked-out windows) Okay, time for an actually useful feature: interactive mouse resizing (yes, finally). This was, as it goes, quite fiddly to implement, especially since niri has to negotiate with the window during the process. I also added a double-resize-click (i.e. trigger a resize twice quickly) gesture to reset the window height or to toggle full width. Suggested by FreeFull on our Matrix and worked out very well! Really starting to feel quite nice with mouse. (still no transactions yet) nvim really taking its time processing all this 1000 Hz worth of resizing lol Since I'm in a mouse gesture mood today: hooked up the horizontal touchpad swipe to Mod + middle mouse drag and omg it feels so good with the spring deceleration and all (of course it also correctly avoids the touchpad scaling, so that when using the mouse gesture, the cursor location is always exactly anchored to the view position) Now for something fun. I'm experimenting with the ability to set custom shaders for animations. Today I added custom shader support for window-close, which lets me make this cool falling down animation! This is entirely optional of course, and there's no performance impact if you don't use it. Also, custom shaders, like the rest of the niri config, are live-reloaded, making it easy to play around with them. Been fixing quite a bit of interactive resize jank and other small stuff since the last time, but also added custom shader support for window-open, thus completing it for all three main window animations (open, close, resize). Now I didn't actually have any good idea of what I might want in a window open custom shader (I like the default), so I made a simple expanding circle animation to showcase it. Niri 0.1.6 with interactive window resizing, rounded corners, named workspaces, mouse view scrolling, animation custom shaders, screen transition! https://github.com/YaLTeR/niri/releases/tag/v0.1.6 Didn't realize quite how many release notes there would be this time; even had to use an extra level of headings. 😅 Over the past few weeks I've been working on fractional scaling for niri. A simple implementation took about a day, but to do it *properly* I had to refactor the entire layout code to work in floating-point. The result is well worth it though. Borders, gaps and windows are always physical-pixel aligned, and not restricted to integer logical pixel positions. There's no blur or position-dependent +-1 px jank. Fractional-scale-aware clients remain crisp at any scale. So it turns out that changing PipeWire screencast stream resolution on the fly is actually not that hard! Which is great news because it's required (or at least very desirable) for implementing window screencasting. Phew, finished the initial implementation of window screencasting in niri! Complete with stream resolution change on window resizing. Some details are still iffy regarding frame timing and frame callbacks, especially to obscured windows, but it *should* work decently fine for now. Icons are missing in the portal dialog because apparently Shell keeps track of Wayland app ID to .desktop file mapping internally and returns the .desktop file name to the portal for it to get the icon. Which is a bit too much effort for me to replicate for now. 😅 Fixed transparency support. Turns out the BGRA format should've been in a separate pod, rather than as a choice in the same pod. Should've looked at Mutter code sooner as usual.. niri v0.1.7 is out with fractional scaling, window screencasts and many smaller improvements! Apparently there's a cool Pango flag that enables subpixel glyph positioning which makes things scale smoother and improves kerning! Thanks Benjamin (the GTK maintainer) for the suggestion Oops, looks like niri had outstarred PaperWM on GitHub 🙈 PaperWM introduced me to (and made me fall in love with) scrollable tiling. It's a solid implementation on top of GNOME, so you get all the benefits of running a stable and well-supported DE (and Xwayland). PaperWM is also a very direct inspiration for many parts of niri! Implemented the thing where you can toggle the mouse pointer on the screenshot after the fact (by pressing P). Gonna add a help panel here soon to remind you of this. One of the niri users has contributed a gradient interpolation color space setting! Now you can do pretty gradient borders in srgb-linear, oklab and oklch (in all four hue directions). I made a COPR for git builds of niri: https://copr.fedorainfracloud.org/coprs/yalter/niri-git Apparently this is a thing you can do; relatively easily, even. It's fully automatic, triggers on a webhook from GitHub. Thanks Michael (my4ng) from our Matrix channel for showing me the relevant docs and doing the initial work! also in an act of severe procrastination i decided to try drawing a diagram of the redraw state machine: https://github.com/YaLTeR/niri/wiki/Redraw-Loop Spent a better part of today, but I've got dmabuf modifier negotiation fully working in my pipewire screencasting code in niri! This happens to finally fix screencasting on NVIDIA GPUs. Still haven't got any GStreamer pipeline working though, maybe needs a pipewire update. Apparently, my AMD selects a BGRA format modifier that has two planes. Some out of band info I suppose. And on Intel a preferred BGRA modifier has three entire planes! My Wayland compositor, niri, turns one today! :ablobcatrave: Here's v0.1.8 with bind key repeat, screenshot UI pointer toggle, gradient color spaces, wlr screencopy v3 and output management, and lots of other improvements: https://github.com/YaLTeR/niri/releases/tag/v0.1.8 omg, my friend drew a birthday nirik :blobcataww: i recorded a demo video for the focus-follows-mouse section to showcase it lol Several days and a lot of hair pulling later, I *think* I got resize transactions working?? Maybe? Hopefully? Resize transactions is when all windows that must resize together, resize together, with no mismatching frames in between. For example, all windows in a column must add up to the full screen height. This requires correct configure acks on the client (looking at you, Blender) and very careful state update delaying and resize throttling on the compositor side. narrator: there was still one case where it didn't work due to a bug okay, surely this time it definitely works for all correctly written clients Here's one mainly for people who disable animations: window closing now runs in a transaction with the other windows resizing. This means, no background flicker. There's been another logo discussion in the niri Matrix room with some quite interesting concepts emerging. Here's one by Endg4me_ with edits by bluelinden and myself, and inspiration from a concept by ElKowar. What do you think? I'm working on an "event stream" IPC for niri where you get notified about events as they happen. For example, "workspace switched" or "keyboard layout changed". To give it a good test, I actually started implementing native niri support in Waybar. You can give it a try too: https://github.com/YaLTeR/niri/pull/453#issuecomment-2317110904 I finished the initial event stream IPC implementation for niri. My Waybar fork implements a decent amount of the modules niri/workspaces, niri/window, niri/language. Please give that a try, also anyone who makes IPC scripts or bars please give a try to the event stream IPC itself so we can find any design flaws before merging. The PR you will need: https://github.com/YaLTeR/niri/pull/453#issuecomment-2321730996 Waybar fork is linked from there. Set up CI rustdoc publishing for niri-ipc: https://yalter.github.io/niri/niri_ipc/ This has the entirety of the niri IPC documented, including the new event stream events. After the Waybar maintainer speedran merging my niri modules and releasing, I would feel bad delaying any longer, so here's niri 0.1.9 :) Event stream IPC for bars, better window resizing, properly named outputs, on-demand VRR, out-of-the-box fix for NVIDIA flickering, and other improvements! Made interactive move work on touch (and resize too while I was at it), didn't need that many changes actually Added a bit of rubberbanding before the window is "dragged out" of the layout. Should help avoid unintended layout changes. Along with a few more fixes I did, I think interactive move should be good to merge? It's not 100% perfect and jank free, but I'm fairly sure I got all the important things done. Will give it some more testing. |
There's actually an important edge case here: if you open the screenshot UI while recording the monitor, then the screenshot UI preview will show the window, and OBS, recording the screenshot UI preview, will hence also show the window. There are trade-offs here for how you want this to work; for now I put a big warning around the option, and added a stricter mode that blocks out the window from ANY screen capture (which means you can't screenshot it).