"Marco Cavenati" <marco.caven...@eurecom.fr> writes: > On Friday, April 11, 2025 14:24 CEST, Fabiano Rosas <faro...@suse.de> wrote: > >> > If bitmap 0 implies zero page, we could call `ram_handle_zero` >> > in `read_ramblock_mapped_ram` for the clear bits. >> > Or do you fear this might be unnecessary expensive for migration? >> >> Yes, unfortunately the peformance difference is noticeable. But we could >> have a slightly different algorithm for savevm. At this point it might >> be easier to just duplicate read_ramblock_mapped_ram(), check for savevm >> in there and see what that the resulting code looks like. > > I tried to get some numbers in a "bad case" scenario restoring a > clean, fully booted, idle Debian VM with 4GB of ram. The zero pages > are ~90%. I'm using a nvme ssd to store the snapshot and I repeated > the restore 10 times with and without zeroing (`ram_handle_zero`). > The restore takes on average +25% of time. > (It's not a broad nor deep investigation.) > > So, I see your point on performance, but I'm not fully comfortable > with the difference in zero page handling between mapped-ram on > and mapped-ram off. In the former case zero pages are skipped, while > in the latter they are explicitly zeroed. > Enabling mapped-ram has the implicit effect to also skip zero-pages. > I think it is an optimization not really bound to mapped-ram and it > could be better to have this feature external to mapped-ram, enabled > when the destination ram is known to be already zeroed (also for > mapped-ram off ideally). >
>From a design perspective that makes sense. The only use case currently would be mapped-ram _migration_ (as opposed to snapshot) because non-mapped-ram migration is usually done live. The stream of pages will potentially have several versions of the same page, therefore we need to clear it when it's a zero page. We'd benefit from a separate "don't load zero pages" option only when the VM is guaranteed to be stopped and more importantly, not allowed to be unstopped. This is the tricky part. We have experimented with and dropped the idea of detecting a stopped-vm-migration for mapped-ram (the idea back then was not to do better zero page handling, but skip dirty tracking altogether). Since we're dealing with snapshots here, which are asynchronous, I'm wondering wheter it would make sense to extend the zero-page-detection option to the destination. Then we could bind the loadvm process to zero-page-detection=none because I don't think we can safely ignore them anyway. >> By the way, what's your overall goal with enabling the feature? Do you >> intent to enable further capabilities for snapshot? Specifically >> multifd. I belive the zero page skip is responsible for most of the >> performance gains for mapped-ram without direct-io and multifd. The >> benefit of bounded stream size doesn't apply to snapshots because >> they're not live. > > My overall goal is a hot-loadvm feature that currently lives in a fork > downstream and has a long way before getting in a mergeable state :) Cool. Don't hesitate to send stuff our way, the sooner and more often we discuss this, the better the chances of getting it merged upstream. Do you use libvirt at all? Mapped-ram support has been added to it in the latest version. The original use-case for mapped-ram was quick snapshot saving and loading after all. Libvirt does it in a way that does not use savevm, which might be helpful. > In a nutshell, I'm using dirty page tracking to load from the snapshot > only the pages that have been dirtied between two loadvm; > mapped-ram is required to seek and read only the dirtied pages. But then you'd have a bitmap of pages you could correlate with the file_bmap and force-load whatever pages you need. It doesn't seem like zero page handling would affect you too much in that case.
