[...] So, I'm not sure if any changes to the status quo actually improve the > current situation, but I'm open to other ideas.
Thanks Michael for the reply, in the meantime I also found that it seems to be no issue in most real world cases - more details on that in the LP bug. I subscribed to the upstream issue 7104 you pointed me to (thanks) but I think for this case I reported here we can close it for now and keep the status-quo for now. I might have a small chrony patch to improve how it works together with systemd-timesyncd, but that is a totally different issue. So please feel free to close this bug unless you are interested to keep it open for further discussion.

