Email or username:

Password:

Forgot your password?
Hector's posts Post Back to profile
Top-level
Nicolás Alvarez

@marcan hm what about sync; time (tar xf linux-6.3.3.tar && sync)?

20 May 2023 at 4:13 | Wall-to-wall | Open on social.treehouse.systems
12 comments
Hector Martin

@nicolas17 Doesn't make much of a difference, the sync after only takes 0.2s or so.

20 May 2023 at 4:15 | Open on social.treehouse.systems
Ariadne Conill 🐰

@marcan @nicolas17 Darwin's VFS is going to have a lot of overhead anyway, I would figure due to the Mach bullshit

20 May 2023 at 4:16 | Open on social.treehouse.systems
Sergey Bugaev

@ariadne @marcan @nicolas17 huh? The VFS does not have anything to do with Mach whatsoever, no? This is not a microkernel system we're talking about.

20 May 2023 at 5:40 | Open on floss.social
Ariadne Conill 🐰

@bugaevc @marcan @nicolas17 every file handle in darwin is still backed by a mach port

20 May 2023 at 5:40 | Open on social.treehouse.systems
Pierre H.
Sergey Bugaev
Ariadne Conill 🐰

@bugaevc @marcan @nicolas17 well, for one, mmap(2) would have to somehow interact with mach ports, given that mach provides the virtual memory system.

i mean, i could be wrong, i'm not terribly interested in spelunking through the Darwin source code at this immediate moment

20 May 2023 at 6:09 | Open on social.treehouse.systems
Ariadne Conill 🐰

@bugaevc @marcan @nicolas17

by extension, however, the point is that any resource that can possibly interact with mach should be backed by resources accessible by mach. e.g. kernel handles of all kinds.

it is, admittedly possible that apple has decided to something far more cursed, like translate resources to be accessible by the mach layer as needed, but this seems far more complicated than just backing all kernel resources by mach ports to begin with

20 May 2023 at 6:18 | Open on social.treehouse.systems
Ariadne Conill 🐰

@bugaevc @marcan @nicolas17

though it seems that you're right that file handles do not normally have mach ports on Darwin (which is surprising to me):

a program which does nothing has 11 ports opened. modify that program to open a file and it still only has 11 ports opened.

🤷

20 May 2023 at 6:27 | Open on social.treehouse.systems
Sergey Bugaev

@ariadne @marcan @nicolas17 there is a semi-public way to wrap an fd into a Mach port (fileport) that you can then send to another process via Mach IPC, and then unwrap to receive an fd in the new process (kind of like SCM_RIGHTS). But this is just that, an explicit wrapper.

20 May 2023 at 6:53 | Open on floss.social
Nicolás Alvarez

@marcan I was thinking it could make the gap *bigger* due to macOS doing more fsync-cheating than Linux :P

20 May 2023 at 4:19 | Open on social.treehouse.systems
Hector Martin

@nicolas17 We also cheat on Linux, we're not stupid, I'm not going to lose to them just because their NVMe firmware has abysmal flush performance :P

But our flush interval is way shorter so we cheat better. You can disable it with a module param if you really care (I can't imagine who would other than people running database servers on Apple Silicon on the internal NVMe?)

Edit: to be clear, the problem is *frequent* NVMe flushes suck on Apple controllers. A single flush at the end is negligible, it's when you have stuff like apt-get flushing on every file that we wound up with pathologically terribad performance on Linux. On Linux we throttle NVMe flushes to 1/sec; macOS just doesn't do them at all when you do a normal fsync, instead they have launchd doing systemwide flushes every 30 seconds or something silly like that, plus that secret nonstandard "no really, flush all the way" fcntl that nobody uses until they actually lose data and learn about it.

@nicolas17 We also cheat on Linux, we're not stupid, I'm not going to lose to them just because their NVMe firmware has abysmal flush performance :P

But our flush interval is way shorter so we cheat better. You can disable it with a module param if you really care (I can't imagine who would other than people running database servers on Apple Silicon on the internal NVMe?)

20 May 2023 at 4:22 | Open on social.treehouse.systems
Go Up