On Thu, Apr 26, 2018 at 11:33:53PM +0000, Eric Wheeler wrote: > Hello all,
Hi, Eric, > > This is my first time inside of the qemu code, so your help is greatly > appreciated! > > I have been experimenting with stop/start of VMs to/from a migration > stream that excludes RAM pages and let the RAM pages come from memory file > provided by the memory-backend-file called '/dev/shm/mem'. > > To disable writing of memory pages to the migration stream, I've disabled > calls to ram_find_and_save_block in ram_save_iterate() and > ram_save_complete() (see patch below). Thus, the migration stream has the > "ram" SaveStateEntry section start/ends, but no pages: > > qemu-system-x86_64 \ > -object > memory-backend-file,prealloc=no,mem-path=/dev/shm/mem,id=ram-node0,host-nodes=0,policy=bind,size=64m,share=on > \ > -numa node,nodeid=0,cpus=0,memdev=ram-node0\ > -m 64 -vnc 0:0 > > Once the VM is running, I press ctrl-B to get the IPXE prompt and then > run 'kernel http://192.168.0.1/foo' to start a network request and watch > it in tcpdump. > > Once the download starts, I save the migration file: > migrate "exec:cat > /dev/shm/t" > > # ls -lh /dev/shm/t > -rw-r--r-- 1 root root 321K Apr 26 16:06 /dev/shm/t > > Now I can kill qemu and boot it again with -incoming: > > qemu-system-x86_64 \ > -object > memory-backend-file,prealloc=no,mem-path=/dev/shm/mem,id=ram-node0,host-nodes=0,policy=bind,size=64m,share=on > \ > -numa node,nodeid=0,cpus=0,memdev=ram-node0\ > -m 64 -vnc 0:0 \ > -incoming 'exec:cat /dev/shm/t' > > It seems to work. That is, network traffic continues (http from IPXE) > which I can see from tcpdump. I can type into the console and it moves > the cursor around---but there is nothing on the screen except the blinking > text-mode cursor! I can even blindly start a new transfer in ipxe: kernel > http://192.168.0.222/foo2 and see it in tcpdump. > > So what am I missing here? Is the video memory not saved to /dev/shm/mem? > > Or perhaps it is saved, but VGA isn't initialized to use what is > already in /dev/shm/mem? I've tried the cirrus, std, and vmware drivers > to see if they behave differently, but the do not seem to. My wild guess is that we might still need to migrate some RAM besides the /dev/shm/mem file. We have at least these ramblocks to migrate: $ ./x86_64-softmmu/qemu-system-x86_64 -monitor stdio -m 2G QEMU 2.12.0 monitor - type 'help' for more information (qemu) info ramblock Block Name PSize Offset Used Total pc.ram 4 KiB 0x0000000000000000 0x0000000080000000 0x0000000080000000 vga.vram 4 KiB 0x0000000080080000 0x0000000001000000 0x0000000001000000 /rom@etc/acpi/tables 4 KiB 0x0000000081100000 0x0000000000020000 0x0000000000200000 pc.bios 4 KiB 0x0000000080000000 0x0000000000040000 0x0000000000040000 0000:00:03.0/e1000.rom 4 KiB 0x00000000810c0000 0x0000000000040000 0x0000000000040000 pc.rom 4 KiB 0x0000000080040000 0x0000000000020000 0x0000000000020000 0000:00:02.0/vga.rom 4 KiB 0x0000000081080000 0x0000000000010000 0x0000000000010000 /rom@etc/table-loader 4 KiB 0x0000000081300000 0x0000000000001000 0x0000000000001000 /rom@etc/acpi/rsdp 4 KiB 0x0000000081340000 0x0000000000001000 0x0000000000001000 And my understanding is that /dev/shm/mem only corresponds to the "pc.ram" entry. I suspect the rest of RAMBlocks will still need to be migrated. For example, the VGA ram. Meanwhile, could I ask about where will this be used? Is there anything to do with something like a "distributed memory cache" that provide memory service across multiple hosts? Best Regards, > > Thanks for your help! > > -- > Eric Wheeler > > > diff --git a/migration/ram.c b/migration/ram.c > index 021d583..9f4bfff 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -2267,9 +2267,9 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) > t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); > i = 0; > while ((ret = qemu_file_rate_limit(f)) == 0) { > - int pages; > + int pages = 0; > > - pages = ram_find_and_save_block(rs, false); > + if (0) pages = ram_find_and_save_block(rs, false); > /* no more pages to sent */ > if (pages == 0) { > done = 1; > @@ -2338,9 +2338,9 @@ static int ram_save_complete(QEMUFile *f, void *opaque) > > /* flush all remaining blocks regardless of rate limiting */ > while (true) { > - int pages; > + int pages = 0; > > - pages = ram_find_and_save_block(rs, !migration_in_colo_state()); > + if (0) pages = ram_find_and_save_block(rs, > !migration_in_colo_state()); > /* no more blocks to sent */ > if (pages == 0) { > break; > > -- Peter Xu
