Chris Siebenmann

Indeed, 'getent passwd|group systemd-timesync' works on all Ubuntu 18.04 machines except the one problem one, even though the system still knows what UID and GID to give the directory. So: left hand not talking to right hand, and it's undebuggable.

Thanks, systemd, for returning us to the era where the best answer to system problems for almost everyone is 'hell, I don't know, reboot the system and hope it goes away'. Can't reboot the system? Tough luck, you're up the creek.

It appears most likely that systemd half-forgot its own DynamicUser(s) when a systemd package update happened and systemd re-exec'd itself. The internals of dynamic users are undocumented, of course, so who knows.

I was wrong about the systemd timesyncd problem, it's not DynamicUser. It's more mysterious and screwed up than that and I have no answers, just more frustrations and a blog rant (and a bunch of strace output because why not, whatever).

The systemd timesync problem appears to be because there are lingering dead xrdp-chansrv FUSE mounts in /proc/mounts, which causes systemd to give up when it's trying to create a new namespace and never switch timesyncd to it, so /var/lib/private access fails.

Systemd does not of course log any sort of failure message when it gives up on setting up the DynamicUser private namespace; it just goes ahead and silently runs the service in the regular filesystem, even though it knows that is guaranteed to fail.

Attempting to manually unmount the nominal FUSE filesystem mounts as root gets the great error: # umount /some/nfs/fs/thinclient_drives umount: /some/nfs/fs/thinclient_drives: block devices are not permitted on filesystem.

Attempting to unmount the FUSE filesystems as the user fails, ultimately with the error 'Operation not permitted' (if one writes a program to just directly call umount(), since umount will try to stat things and fail with various charming errors due to that).