Re: virtio: Add memory statistics reporting to the balloon driver (V4)

2009-12-01 Thread Avi Kivity
On 12/01/2009 05:05 PM, Adam Litke wrote: > On Tue, 2009-12-01 at 16:56 +0200, Avi Kivity wrote: > >> What about a spec update? Did that happen and I just missed it? >> > Yes it did. I forgot to update this patch leader note with the link: > http://ozlabs.org/~rusty/virtio-spec/virtio-s

Re: virtio: Add memory statistics reporting to the balloon driver (V4)

2009-12-01 Thread Adam Litke
On Tue, 2009-12-01 at 16:56 +0200, Avi Kivity wrote: > What about a spec update? Did that happen and I just missed it? Yes it did. I forgot to update this patch leader note with the link: http://ozlabs.org/~rusty/virtio-spec/virtio-spec-0.8.2.pdf -- Thanks, Adam _

Re: virtio: Add memory statistics reporting to the balloon driver (V4)

2009-12-01 Thread Avi Kivity
On 11/30/2009 06:14 PM, Adam Litke wrote: > Changes since V3: > - Do not do endian conversions as they will be done in the host > - Report stats that reference a quantity of memory in bytes > - Minor coding style updates > > Changes since V2: > - Increase stat field size to 64 bits > - Re

Re: virtio: Add memory statistics reporting to the balloon driver (V4)

2009-12-01 Thread Adam Litke
Thanks Rusty. Acked-by: Adam Litke On Tue, 2009-12-01 at 12:54 +1030, Rusty Russell wrote: > On Tue, 1 Dec 2009 02:44:15 am Adam Litke wrote: > > Changes since V3: > > OK, I applied this. And here's the fixes I applied afterwards (to save YA > round trip: please ack and I'll fold them together

Re: virtio: Add memory statistics reporting to the balloon driver (V4)

2009-11-30 Thread Rusty Russell
On Tue, 1 Dec 2009 02:44:15 am Adam Litke wrote: > Changes since V3: OK, I applied this. And here's the fixes I applied afterwards (to save YA round trip: please ack and I'll fold them together) virtio: balloon fixes 1) Tag and val args to update_stat() are no longer __le. 2) pages_to_bytes() s

virtio: Add memory statistics reporting to the balloon driver (V4)

2009-11-30 Thread Adam Litke
Changes since V3: - Do not do endian conversions as they will be done in the host - Report stats that reference a quantity of memory in bytes - Minor coding style updates Changes since V2: - Increase stat field size to 64 bits - Report all sizes in kb (not pages) - Drop anon_pages stat and f

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-23 Thread Vadim Rozenfeld
On 11/23/2009 01:00 PM, Dor Laor wrote: > On 11/23/2009 11:44 AM, Michael S. Tsirkin wrote: >> On Thu, Nov 19, 2009 at 09:19:05AM -0600, Adam Litke wrote: >>> Rusty and Anthony, >>> If I've addressed all outstanding issues, please consider this patch >>> for >>> inclusion. Thanks. >>> >>> Changes

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-23 Thread Dor Laor
On 11/23/2009 11:44 AM, Michael S. Tsirkin wrote: > On Thu, Nov 19, 2009 at 09:19:05AM -0600, Adam Litke wrote: >> Rusty and Anthony, >> If I've addressed all outstanding issues, please consider this patch for >> inclusion. Thanks. >> >> Changes since V2: >> - Increase stat field size to 64 bits

Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-23 Thread Michael S. Tsirkin
On Thu, Nov 19, 2009 at 09:19:05AM -0600, Adam Litke wrote: > Rusty and Anthony, > If I've addressed all outstanding issues, please consider this patch for > inclusion. Thanks. > > Changes since V2: > - Increase stat field size to 64 bits > - Report all sizes in kb (not pages) > - Drop anon_pa

Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-19 Thread Rusty Russell
On Fri, 20 Nov 2009 01:49:05 am Adam Litke wrote: > Rusty and Anthony, > If I've addressed all outstanding issues, please consider this patch for > inclusion. Thanks. > > Changes since V2: > - Increase stat field size to 64 bits > - Report all sizes in kb (not pages) Hi Adam, Looks like we

Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-19 Thread Adam Litke
On Thu, 2009-11-19 at 18:13 +0200, Avi Kivity wrote: > On 11/19/2009 05:58 PM, Adam Litke wrote: > > On Thu, 2009-11-19 at 17:22 +0200, Avi Kivity wrote: > > > >> On 11/19/2009 05:19 PM, Adam Litke wrote: > >> > >>> Rusty and Anthony, > >>> If I've addressed all outstanding issues, please

Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-19 Thread Avi Kivity
On 11/19/2009 05:58 PM, Adam Litke wrote: > On Thu, 2009-11-19 at 17:22 +0200, Avi Kivity wrote: > >> On 11/19/2009 05:19 PM, Adam Litke wrote: >> >>> Rusty and Anthony, >>> If I've addressed all outstanding issues, please consider this patch for >>> inclusion. Thanks. >>> >>> +struct vi

Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-19 Thread Adam Litke
On Thu, 2009-11-19 at 17:22 +0200, Avi Kivity wrote: > On 11/19/2009 05:19 PM, Adam Litke wrote: > > Rusty and Anthony, > > If I've addressed all outstanding issues, please consider this patch for > > inclusion. Thanks. > > > > +struct virtio_balloon_stat > > +{ > > + __le16 tag; > > + __le64

Re: virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-19 Thread Avi Kivity
On 11/19/2009 05:19 PM, Adam Litke wrote: > Rusty and Anthony, > If I've addressed all outstanding issues, please consider this patch for > inclusion. Thanks. > > +struct virtio_balloon_stat > +{ > + __le16 tag; > + __le64 val; > +}; > + > You're not doing endian conversion in the hos

virtio: Add memory statistics reporting to the balloon driver (V3)

2009-11-19 Thread Adam Litke
Rusty and Anthony, If I've addressed all outstanding issues, please consider this patch for inclusion. Thanks. Changes since V2: - Increase stat field size to 64 bits - Report all sizes in kb (not pages) - Drop anon_pages stat and fix endianness conversion Changes since V1: - Use a virtqueue

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver (V2)

2009-11-18 Thread Rusty Russell
On Thu, 19 Nov 2009 01:32:26 am Anthony Liguori wrote: > Rusty Russell wrote: > > The little-endian conversion of the balloon driver is a historical mistake > > (no other driver does this). Let's not extend it to the stats. > > I think the mistake is that the other drivers don't do that. > > We

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver (V2)

2009-11-18 Thread Jamie Lokier
Anthony Liguori wrote: > Rusty Russell wrote: > >The little-endian conversion of the balloon driver is a historical mistake > >(no other driver does this). Let's not extend it to the stats. > > I think the mistake is that the other drivers don't do that. > > We cheat in qemu and assume that the

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver (V2)

2009-11-18 Thread Anthony Liguori
Rusty Russell wrote: > On Wed, 18 Nov 2009 07:06:29 am Adam Litke wrote: > >> virtio: Add memory statistics reporting to the balloon driver (V2) >> >> Changes since V1: >> - Use a virtqueue instead of the device config space >> > > Hi Adam, >

Re: virtio: Add memory statistics reporting to the balloon driver (V2)

2009-11-17 Thread Rusty Russell
On Wed, 18 Nov 2009 07:06:29 am Adam Litke wrote: > virtio: Add memory statistics reporting to the balloon driver (V2) > > Changes since V1: > - Use a virtqueue instead of the device config space Hi Adam, If Anthony's happy, I'm happy with this approach. Couple of mi

virtio: Add memory statistics reporting to the balloon driver (V2)

2009-11-17 Thread Adam Litke
virtio: Add memory statistics reporting to the balloon driver (V2) Changes since V1: - Use a virtqueue instead of the device config space When using ballooning to manage overcommitted memory on a host, a system for guests to communicate their memory usage to the host can provide information

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-11 Thread Rusty Russell
On Thu, 12 Nov 2009 01:38:34 am Adam Litke wrote: > > But it raises the question: what stats are generally useful cross-OS? > > Should > > we be supplying numbers like "unused" (free) "instantly discardable" (ie. > > clean), "discardable to disk" (ie. file-backed), "discardable to swap" > > (ie.

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-11 Thread Adam Litke
On Wed, 2009-11-11 at 13:13 +1030, Rusty Russell wrote: > > It's not laziness, it's consistency. How is actual different than free > > memory or any other stat? > > Because it's a COLLECTION of stats. For example, swap in should be < swap > out. Now, the current Linux implementation of all_vm_

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-11 Thread Avi Kivity
On 11/11/2009 03:26 PM, Adam Litke wrote: > On Wed, 2009-11-11 at 10:12 +, Daniel P. Berrange wrote: > >> This all suggests that we should only update the stats from the guest >> when something on the host actually asks for them by issuing the QEMU >> monitor command. We don't want any kind

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-11 Thread Adam Litke
On Wed, 2009-11-11 at 10:12 +, Daniel P. Berrange wrote: > This all suggests that we should only update the stats from the guest > when something on the host actually asks for them by issuing the QEMU > monitor command. We don't want any kind of continuous polling of stats > at any frequency, i

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-11 Thread Daniel P. Berrange
On Wed, Nov 11, 2009 at 09:24:09AM +, Jamie Lokier wrote: > Anthony Liguori wrote: > > Avi Kivity wrote: > > >On 11/10/2009 04:36 PM, Anthony Liguori wrote: > > >> > > >>>A stats vq might solve this more cleanly? > > >> > > >>actual and target are both really just stats. Had we implemented >

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-11 Thread Jamie Lokier
Anthony Liguori wrote: > Avi Kivity wrote: > >On 11/10/2009 04:36 PM, Anthony Liguori wrote: > >> > >>>A stats vq might solve this more cleanly? > >> > >>actual and target are both really just stats. Had we implemented > >>those with a vq, I'd be inclined to agree with you but since they're > >>

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Rusty Russell
On Wed, 11 Nov 2009 10:37:56 am Anthony Liguori wrote: > Rusty Russell wrote: > > You register an outbuf at initialization time. The host hands it back when > > it wants you to refill it with stats. > > That's strangely backwards. Guest send a stat buffer that's filled out, > host acks it when

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Anthony Liguori
Rusty Russell wrote: > On Wed, 11 Nov 2009 08:22:42 am Anthony Liguori wrote: > >> Rusty Russell wrote: >> >>> On Tue, 10 Nov 2009 03:02:06 am Adam Litke wrote: >>> >>> A simpler approach is to collect memory statistics in the virtio balloon driver and communicate them t

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Rusty Russell
On Wed, 11 Nov 2009 08:22:42 am Anthony Liguori wrote: > Rusty Russell wrote: > > On Tue, 10 Nov 2009 03:02:06 am Adam Litke wrote: > > > >> A simpler approach is to collect memory statistics in the virtio > >> balloon driver and communicate them to the host via the device config > >> space. >

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Rusty Russell
On Wed, 11 Nov 2009 01:06:14 am Anthony Liguori wrote: > Rusty Russell wrote: > > On Tue, 10 Nov 2009 03:02:06 am Adam Litke wrote: > > > >> A simpler approach is to collect memory statistics in the virtio > >> balloon driver and communicate them to the host via the device config > >> space. >

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Anthony Liguori
Rusty Russell wrote: > On Tue, 10 Nov 2009 03:02:06 am Adam Litke wrote: > >> A simpler approach is to collect memory statistics in the virtio >> balloon driver and communicate them to the host via the device config space. >> > > There are two issues I see with this. First, there's an atom

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Anthony Liguori
Avi Kivity wrote: > On 11/10/2009 04:36 PM, Anthony Liguori wrote: >> >>> A stats vq might solve this more cleanly? >> >> actual and target are both really just stats. Had we implemented >> those with a vq, I'd be inclined to agree with you but since they're >> implemented in the config space, i

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Avi Kivity
On 11/10/2009 04:36 PM, Anthony Liguori wrote: > >> A stats vq might solve this more cleanly? > > actual and target are both really just stats. Had we implemented > those with a vq, I'd be inclined to agree with you but since they're > implemented in the config space, it seems natural to extend

Re: [Qemu-devel] Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-10 Thread Anthony Liguori
Rusty Russell wrote: > On Tue, 10 Nov 2009 03:02:06 am Adam Litke wrote: > >> A simpler approach is to collect memory statistics in the virtio >> balloon driver and communicate them to the host via the device config space. >> > > There are two issues I see with this. First, there's an atom

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-09 Thread Rusty Russell
On Tue, 10 Nov 2009 03:02:06 am Adam Litke wrote: > A simpler approach is to collect memory statistics in the virtio > balloon driver and communicate them to the host via the device config space. There are two issues I see with this. First, there's an atomicity problem since you can't tell when t

virtio: Add memory statistics reporting to the balloon driver

2009-11-09 Thread Adam Litke
When using ballooning to manage overcommitted memory on a host, a system for guests to communicate their memory usage to the host can provide information that will minimize the impact of ballooning on the guests. The current method employs a daemon running in each guest that communicates memory st

Re: virtio: Add memory statistics reporting to the balloon driver

2009-11-05 Thread Anthony Liguori
a...@linux.vnet.ibm.com wrote: > Here are the corresponding changes to the Linux virtio driver... > > virtio: Add memory statistics reporting to the balloon driver > > When using ballooning to manage overcommitted memory on a host, a system > for > guests