Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 03/02/2011 11:30 AM, Avi Kivity wrote: It's really the natural generalization of what you're proposing. So basically, the only differences are: 1) always use the new RAID1 format 2) drop the progress bitmap 3) support multiple devices per file 4) let drive properties be specified beyond

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 03/01/2011 05:51 PM, Anthony Liguori wrote: Do a hot unplug of a network device with upstream libvirt with acpiphp unloaded, consult libvirt and then consult the monitor to see who has the right view of the guests config. libvirt is right and the monitor is wrong. On real hardware, calli

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 03/02/2011 08:00 AM, Avi Kivity wrote: On 03/02/2011 02:39 PM, Anthony Liguori wrote: Here is where your race is: 2. Management sends a switch command 3. QEMU receives switch command 4. QEMU stops doubling IO and switches to the destination 5. QEMU sends acknowledgement of switch command

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 03/02/2011 02:39 PM, Anthony Liguori wrote: Here is where your race is: 2. Management sends a switch command 3. QEMU receives switch command 4. QEMU stops doubling IO and switches to the destination 5. QEMU sends acknowledgement of switch command 6. Management receives acknowledge of swi

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 03/01/2011 03:59 AM, Dor Laor wrote: On 02/28/2011 08:12 PM, Anthony Liguori wrote: On Feb 28, 2011 11:47 AM, "Avi Kivity" mailto:a...@redhat.com>> wrote: > > On 02/28/2011 07:33 PM, Anthony Liguori wrote: >> >> >> > >> > You're just ignoring what I've written. >> >> No, you're just impervio

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 03/01/2011 05:51 PM, Anthony Liguori wrote: On 03/01/2011 04:39 AM, Avi Kivity wrote: On 02/28/2011 08:12 PM, Anthony Liguori wrote: On Feb 28, 2011 11:47 AM, "Avi Kivity" mailto:a...@redhat.com>> wrote: > > On 02/28/2011 07:33 PM, Anthony Liguori wrote: >> >> >> > >> > You're just ignorin

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 03/01/2011 04:39 AM, Avi Kivity wrote: On 02/28/2011 08:12 PM, Anthony Liguori wrote: On Feb 28, 2011 11:47 AM, "Avi Kivity" > wrote: > > On 02/28/2011 07:33 PM, Anthony Liguori wrote: >> >> >> > >> > You're just ignoring what I've written. >> >> No, you're just imp

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/28/2011 08:56 PM, Marcelo Tosatti wrote: > > Something that avoids the whole state thing altogether: > > - instead of atomically switching when live copy is done, keep on > issuing writes to both the origin and the live copy > - issue a notification to management > - management receive

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/28/2011 08:12 PM, Anthony Liguori wrote: On Feb 28, 2011 11:47 AM, "Avi Kivity" > wrote: > > On 02/28/2011 07:33 PM, Anthony Liguori wrote: >> >> >> > >> > You're just ignoring what I've written. >> >> No, you're just impervious to my subtle attempt to refocus the

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/28/2011 08:12 PM, Anthony Liguori wrote: On Feb 28, 2011 11:47 AM, "Avi Kivity" mailto:a...@redhat.com>> wrote: > > On 02/28/2011 07:33 PM, Anthony Liguori wrote: >> >> >> > >> > You're just ignoring what I've written. >> >> No, you're just impervious to my subtle attempt to refocu

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Mon, Feb 28, 2011 at 07:47:13PM +0200, Avi Kivity wrote: > On 02/28/2011 07:33 PM, Anthony Liguori wrote: > > > >> > >> You're just ignoring what I've written. > > > >No, you're just impervious to my subtle attempt to refocus the > >discussion on solving a practical problem. > > > >There's a lot

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Feb 28, 2011 11:47 AM, "Avi Kivity" wrote: > > On 02/28/2011 07:33 PM, Anthony Liguori wrote: >> >> >> > >> > You're just ignoring what I've written. >> >> No, you're just impervious to my subtle attempt to refocus the discussion on solving a practical problem. >> >> There's a lot of good, reas

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/28/2011 07:33 PM, Anthony Liguori wrote: > > You're just ignoring what I've written. No, you're just impervious to my subtle attempt to refocus the discussion on solving a practical problem. There's a lot of good, reasonably straight forward changes we can make that have a high return

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Feb 28, 2011 7:21 AM, "Avi Kivity" wrote: > > On 02/28/2011 02:45 PM, Anthony Liguori wrote: >> >> On 02/28/2011 02:38 AM, Avi Kivity wrote: We don't separate configuration from guest state today. Instead of setting ourselves up for failure by setting an unrealistic standard tha

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/28/2011 02:45 PM, Anthony Liguori wrote: On 02/28/2011 02:38 AM, Avi Kivity wrote: We don't separate configuration from guest state today. Instead of setting ourselves up for failure by setting an unrealistic standard that we try to achieve and never do, let's embrace the system that

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/28/2011 02:38 AM, Avi Kivity wrote: We don't separate configuration from guest state today. Instead of setting ourselves up for failure by setting an unrealistic standard that we try to achieve and never do, let's embrace the system that is working for us today. We are authoritative f

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 07:25 PM, Anthony Liguori wrote: On 02/27/2011 10:02 AM, Dor Laor wrote: On 02/27/2011 03:49 PM, Anthony Liguori wrote: On 02/27/2011 03:55 AM, Dor Laor wrote: What about a simpler approach were QMP events will be written to a event-log-file (or even named pipe). The management

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 07:41 PM, Anthony Liguori wrote: I agree 100% the management tool cannot be the authoritative source of state. My position is: - the management tool should be 100% in control of configuration (how the guest is put together from its components) - qemu should be 100% in control o

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 09:31 AM, Avi Kivity wrote: On 02/27/2011 04:00 PM, Anthony Liguori wrote: On 02/27/2011 03:10 AM, Avi Kivity wrote: On 02/24/2011 07:58 PM, Anthony Liguori wrote: If you move the cdrom to a different IDE channel, you have to update the stateful non-config file. Whereas if you

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 10:02 AM, Dor Laor wrote: On 02/27/2011 03:49 PM, Anthony Liguori wrote: On 02/27/2011 03:55 AM, Dor Laor wrote: What about a simpler approach were QMP events will be written to a event-log-file (or even named pipe). The management tool can just use a small daemon that does noth

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 03:49 PM, Anthony Liguori wrote: On 02/27/2011 03:55 AM, Dor Laor wrote: What about a simpler approach were QMP events will be written to a event-log-file (or even named pipe). The management tool can just use a small daemon that does nothing other than write QMP events to a log.

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 04:00 PM, Anthony Liguori wrote: On 02/27/2011 03:10 AM, Avi Kivity wrote: On 02/24/2011 07:58 PM, Anthony Liguori wrote: If you move the cdrom to a different IDE channel, you have to update the stateful non-config file. Whereas if you do $ qemu-img create -f cd-tray -b ~/fo

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 03:10 AM, Avi Kivity wrote: On 02/24/2011 07:58 PM, Anthony Liguori wrote: If you move the cdrom to a different IDE channel, you have to update the stateful non-config file. Whereas if you do $ qemu-img create -f cd-tray -b ~/foo.img ~/foo-media-tray.img $ qemu -cdrom ~/fo

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 03:55 AM, Dor Laor wrote: What about a simpler approach were QMP events will be written to a event-log-file (or even named pipe). The management tool can just use a small daemon that does nothing other than write QMP events to a log. There's no real need for this code to live i

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/27/2011 11:10 AM, Avi Kivity wrote: On 02/24/2011 07:58 PM, Anthony Liguori wrote: If you move the cdrom to a different IDE channel, you have to update the stateful non-config file. Whereas if you do $ qemu-img create -f cd-tray -b ~/foo.img ~/foo-media-tray.img $ qemu -cdrom ~/foo-media

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 07:45 PM, Anthony Liguori wrote: Yeah, it feels like we're introducing QEMU level RAID. We already did, with sheepdog (well the RAID code is really outside qemu itself). At what point are we going to add RAID5 support and not just RAID1. And it certainly begs the question o

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 07:58 PM, Anthony Liguori wrote: If you move the cdrom to a different IDE channel, you have to update the stateful non-config file. Whereas if you do $ qemu-img create -f cd-tray -b ~/foo.img ~/foo-media-tray.img $ qemu -cdrom ~/foo-media-tray.img the cd-rom tray state wil

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Wed, Feb 23, 2011 at 3:31 PM, Avi Kivity wrote: > On 02/23/2011 04:35 PM, Anthony Liguori wrote: >> >> QEMU uses the state database to store information that is created >> dynamically.  For instance, devices added through device_add.  A device >> added via -device wouldn't necessary get added t

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 09:22 AM, Avi Kivity wrote: On 02/24/2011 05:00 PM, Anthony Liguori wrote: On 02/24/2011 02:54 AM, Avi Kivity wrote: On 02/23/2011 10:18 PM, Anthony Liguori wrote: Then the management stack has to worry about yet another way of interacting via qemu. { 'StateItem': { 'key': 'st

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 10:39 AM, Marcelo Tosatti wrote: On Thu, Feb 24, 2011 at 05:28:20PM +0200, Avi Kivity wrote: On 02/24/2011 05:14 PM, Marcelo Tosatti wrote: >>The problem with qemu config files is that it splits the >>authoritative source of where images are stored into two.

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 06:39 PM, Marcelo Tosatti wrote: > > > >You can live migrate (but not live migrate with live block migration) > >with live copy in progress, its just that its not supported yet. > > A RAID-1 driver will work with block live migration too. Nobody cares about that one (block copy

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Thu, Feb 24, 2011 at 05:28:20PM +0200, Avi Kivity wrote: > On 02/24/2011 05:14 PM, Marcelo Tosatti wrote: > >> >> The problem with qemu config files is that it splits the > >> >> authoritative source of where images are stored into two. Is it in > >> >> the management tool's database o

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 05:14 PM, Marcelo Tosatti wrote: > >> The problem with qemu config files is that it splits the > >> authoritative source of where images are stored into two. Is it in > >> the management tool's database or is it in qemu's config file? > >> > >> For the problem at hand,

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 05:00 PM, Anthony Liguori wrote: On 02/24/2011 02:54 AM, Avi Kivity wrote: On 02/23/2011 10:18 PM, Anthony Liguori wrote: Then the management stack has to worry about yet another way of interacting via qemu. { 'StateItem': { 'key': 'str', 'value': 'str' } } { 'StateSection': {

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Wed, Feb 23, 2011 at 03:41:30PM -0600, Anthony Liguori wrote: > On 02/23/2011 02:44 PM, Marcelo Tosatti wrote: > > > >>Any indirect qemu state. Block migration is an example, but other > >>examples would be VNC server information (like current password), > >>WCE setting (depending on whether we

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Thu, Feb 24, 2011 at 10:58:10AM +0200, Avi Kivity wrote: > On 02/23/2011 07:49 PM, Marcelo Tosatti wrote: > >On Wed, Feb 23, 2011 at 03:01:14PM +0200, Avi Kivity wrote: > >> On 02/23/2011 01:14 AM, Anthony Liguori wrote: > >> > > >> >-drive already ties into the qemuopts infrastructure and we

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/24/2011 02:54 AM, Avi Kivity wrote: On 02/23/2011 10:18 PM, Anthony Liguori wrote: Then the management stack has to worry about yet another way of interacting via qemu. { 'StateItem': { 'key': 'str', 'value': 'str' } } { 'StateSection': { 'kind': 'str', 'name': 'str', 'items': [ 'State

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

Anthony Liguori writes: > On 02/23/2011 11:26 AM, Markus Armbruster wrote: >>> I don't think it's reasonable to have three different ways to interact >>> with qemu, all needed: the command line, reading and writing the >>> stateful config file, and the monitor. I'd rather push for starting >>> q

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 07:49 PM, Marcelo Tosatti wrote: On Wed, Feb 23, 2011 at 03:01:14PM +0200, Avi Kivity wrote: > On 02/23/2011 01:14 AM, Anthony Liguori wrote: > > > >-drive already ties into the qemuopts infrastructure and we have > >readconfig and writeconfig. I don't think we're missing any m

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 10:18 PM, Anthony Liguori wrote: Then the management stack has to worry about yet another way of interacting via qemu. { 'StateItem': { 'key': 'str', 'value': 'str' } } { 'StateSection': { 'kind': 'str', 'name': 'str', 'items': [ 'StateItem' ] } } { 'StateInfo': { 'sections': [

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

Anthony Liguori writes: > On 02/23/2011 11:18 AM, Avi Kivity wrote: >> On 02/23/2011 06:28 PM, Anthony Liguori wrote: >>> > Well specifically, it has to ask QEMU and QEMU can tell it the > current state via a nice structured data format over QMP. It's a > hell of a lot easier than th

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 02:44 PM, Marcelo Tosatti wrote: Any indirect qemu state. Block migration is an example, but other examples would be VNC server information (like current password), WCE setting (depending on whether we modelled eeprom for the drivers), and persisted device settings (lots of devic

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Wed, Feb 23, 2011 at 02:18:31PM -0600, Anthony Liguori wrote: > On 02/23/2011 11:18 AM, Avi Kivity wrote: > >On 02/23/2011 06:28 PM, Anthony Liguori wrote: > >> > Well specifically, it has to ask QEMU and QEMU can tell it > the current state via a nice structured data format over > Q

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 11:18 AM, Avi Kivity wrote: On 02/23/2011 06:28 PM, Anthony Liguori wrote: Well specifically, it has to ask QEMU and QEMU can tell it the current state via a nice structured data format over QMP. It's a hell of a lot easier than the management tool trying to do this outside of

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 11:26 AM, Markus Armbruster wrote: I don't think it's reasonable to have three different ways to interact with qemu, all needed: the command line, reading and writing the stateful config file, and the monitor. I'd rather push for starting qemu with a blank guest and assembling (col

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Wed, Feb 23, 2011 at 03:01:14PM +0200, Avi Kivity wrote: > On 02/23/2011 01:14 AM, Anthony Liguori wrote: > > > >-drive already ties into the qemuopts infrastructure and we have > >readconfig and writeconfig. I don't think we're missing any major > >pieces to do this in a more proper fashion. >

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

Avi Kivity writes: > On 02/23/2011 04:35 PM, Anthony Liguori wrote: >> On 02/23/2011 07:01 AM, Avi Kivity wrote: >>> On 02/23/2011 01:14 AM, Anthony Liguori wrote: -drive already ties into the qemuopts infrastructure and we have readconfig and writeconfig. I don't think we're miss

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 06:28 PM, Anthony Liguori wrote: Well specifically, it has to ask QEMU and QEMU can tell it the current state via a nice structured data format over QMP. It's a hell of a lot easier than the management tool trying to do this outside of QEMU. So, if qemu crashes, the managemen

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 10:17 AM, Peter Maydell wrote: On 23 February 2011 16:01, Anthony Liguori wrote: Yes. I view the command line as optional. To me, this is the ideal interaction: 1) start qemu with an empty stateful config file 2) issue monitor commands to create all devices and backends

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 10:14 AM, Avi Kivity wrote: On 02/23/2011 06:01 PM, Anthony Liguori wrote: Qemu does keep state. Currently only images, but in theory also the on-board NVRAM. Yeah, this is a good example of an area where a "stateful config file" would be useful. I like the idea of storing

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 23 February 2011 16:01, Anthony Liguori wrote: > Yes.   I view the command line as optional.  To me, this is the ideal > interaction: > > 1) start qemu with an empty stateful config file > > 2) issue monitor commands to create all devices and backends > > 3) the stateful config file totally cap

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 06:01 PM, Anthony Liguori wrote: Qemu does keep state. Currently only images, but in theory also the on-board NVRAM. Yeah, this is a good example of an area where a "stateful config file" would be useful. I like the idea of storing this sort of thing in a text file with a

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 09:31 AM, Avi Kivity wrote: On 02/23/2011 04:35 PM, Anthony Liguori wrote: On 02/23/2011 07:01 AM, Avi Kivity wrote: On 02/23/2011 01:14 AM, Anthony Liguori wrote: -drive already ties into the qemuopts infrastructure and we have readconfig and writeconfig. I don't think we're

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 04:35 PM, Anthony Liguori wrote: On 02/23/2011 07:01 AM, Avi Kivity wrote: On 02/23/2011 01:14 AM, Anthony Liguori wrote: -drive already ties into the qemuopts infrastructure and we have readconfig and writeconfig. I don't think we're missing any major pieces to do this in a m

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 07:01 AM, Avi Kivity wrote: On 02/23/2011 01:14 AM, Anthony Liguori wrote: -drive already ties into the qemuopts infrastructure and we have readconfig and writeconfig. I don't think we're missing any major pieces to do this in a more proper fashion. The problem with qemu conf

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/23/2011 01:14 AM, Anthony Liguori wrote: -drive already ties into the qemuopts infrastructure and we have readconfig and writeconfig. I don't think we're missing any major pieces to do this in a more proper fashion. The problem with qemu config files is that it splits the authoritativ

Re: [Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/22/2011 11:11 PM, Anthony Liguori wrote: The commit file is considered reliable storage for the result of image switch operation. Think of the following scenario: - mgmt app requests live copy of guests ide1-hd0 from /a/image.img to /b/image.img. - mgmt app dies. - guest switches to /b/ima

[Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/22/2011 05:09 PM, Marcelo Tosatti wrote: On Tue, Feb 22, 2011 at 03:11:06PM -0600, Anthony Liguori wrote: On 02/22/2011 03:07 PM, Marcelo Tosatti wrote: On Tue, Feb 22, 2011 at 02:50:09PM -0600, Anthony Liguori wrote: +static int write_commit_file(BdrvCopyState *s) +{ +

[Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Tue, Feb 22, 2011 at 03:11:06PM -0600, Anthony Liguori wrote: > On 02/22/2011 03:07 PM, Marcelo Tosatti wrote: > >On Tue, Feb 22, 2011 at 02:50:09PM -0600, Anthony Liguori wrote: > >>>+static int write_commit_file(BdrvCopyState *s) > >>>+{ > >>>+char commit_msg[1400]; > >>>+const char *b

[Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/22/2011 03:07 PM, Marcelo Tosatti wrote: On Tue, Feb 22, 2011 at 02:50:09PM -0600, Anthony Liguori wrote: +static int write_commit_file(BdrvCopyState *s) +{ +char commit_msg[1400]; +const char *buf = commit_msg; +int len, ret; + +sprintf(commit_msg, "commit QEMU block_c

[Qemu-devel] Re: [patch 2/3] Add support for live block copy

On Tue, Feb 22, 2011 at 02:50:09PM -0600, Anthony Liguori wrote: > >+static int write_commit_file(BdrvCopyState *s) > >+{ > >+char commit_msg[1400]; > >+const char *buf = commit_msg; > >+int len, ret; > >+ > >+sprintf(commit_msg, "commit QEMU block_copy %s -> %s\n", > >s->src_devi

[Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/22/2011 11:00 AM, Marcelo Tosatti wrote: This patch is large and not properly inlined. Can it be split up into more reviewable chunks? Regards, Anthony Liguori

[Qemu-devel] Re: [patch 2/3] Add support for live block copy

On 02/22/2011 11:00 AM, Marcelo Tosatti wrote: Support live image copy + switch. That is, copy an image backing a guest hard disk to a destination image (destination image must be created separately), and switch to this copy. Command syntax: block_copy device filename [commit_filename] [-i] --