Re: [Qemu-devel] [PATCH 10/23] userfaultfd: add new syscall to provide memory externalization

2015-06-24 Thread Dave Hansen
On 05/14/2015 10:31 AM, Andrea Arcangeli wrote: > +static int userfaultfd_wake_function(wait_queue_t *wq, unsigned mode, > + int wake_flags, void *key) > +{ > + struct userfaultfd_wake_range *range = key; > + int ret; > + struct userfaultfd_wait_queue *u

Re: [Qemu-devel] [PATCH 1/2] sched: add virt sched domain for the guest

2012-05-23 Thread Dave Hansen
On 05/23/2012 01:48 AM, Peter Zijlstra wrote: > On Wed, 2012-05-23 at 16:34 +0800, Liu ping fan wrote: >> > so we need to migrate some of vcpus from node-B to node-A, or to >> > node-C. > This is absolutely broken, you cannot do that. > > A guest task might want to be node affine, it looks at the

[Qemu-devel] [PATCH] set VIRTIO_BALLOON_F_MUST_TELL_HOST unconditionally

2011-04-15 Thread Dave Hansen
nything_. This patch makes the "tell host first" logic the only case. This should make everybody happy, and reduce the amount of untested or untestable code in the kernel. Signed-off-by: Dave Hansen Cc: Amit Shah Cc: Anthony Liguori --- hw/virtio-balloon.c |4 1 files ch

Re: [Qemu-devel] [PATCH] set VIRTIO_BALLOON_F_MUST_TELL_HOST unconditionally

2011-04-15 Thread Dave Hansen
On Fri, 2011-04-15 at 11:21 -0500, Anthony Liguori wrote: > > This patch makes the "tell host first" logic the only case. This > > should make everybody happy, and reduce the amount of untested or > > untestable code in the kernel. > > It doesn't make me happy. Darn. > Why would we do this in Q

Re: [Qemu-devel] [PATCH] set VIRTIO_BALLOON_F_MUST_TELL_HOST unconditionally

2011-04-15 Thread Dave Hansen
On Fri, 2011-04-15 at 12:17 -0500, Anthony Liguori wrote: > On 04/15/2011 11:36 AM, Dave Hansen wrote: > >> Why would we do this in QEMU? This prevents the guest from doing > >> ballooning reclaim during OOM. > > What the heck is "ballooning reclaim"? Coul

Re: [Qemu-devel] [PATCH] set VIRTIO_BALLOON_F_MUST_TELL_HOST unconditionally

2011-04-18 Thread Dave Hansen
On Fri, 2011-04-15 at 14:20 -0500, Anthony Liguori wrote: > On 04/15/2011 02:15 PM, Dave Hansen wrote: > > On Fri, 2011-04-15 at 12:17 -0500, Anthony Liguori wrote: > >> If you're in OOM and you need memory, you can't ask the host for more > >> and wait fo

Re: [Qemu-devel] [PATCH 00/10] RFC: userfault

2014-07-03 Thread Dave Hansen
On 07/02/2014 09:50 AM, Andrea Arcangeli wrote: > The MADV_USERFAULT feature should be generic enough that it can > provide the userfaults to the Android volatile range feature too, on > access of reclaimed volatile pages. Maybe. I certainly can't keep track of all the versions of the variations

Re: [Qemu-devel] XSAVES in GET_SUPPORTED_CPUID (was Re: [PATCH] target-i386: add Skylake-Client cpu mode)

2016-05-03 Thread Dave Hansen
On 05/03/2016 10:44 AM, Xiao Guangrong wrote: >>> This is the reason why setup_clear_cpu_cap(X86_FEATURE_XSAVES) >>> introduced by commit e88221c50 >>> caused XSAVES unreported by KVM. >> >> So, is this the right behavior, or KVM can support exposing >> XSAVES to guests even if the cpu_cap bit is c

Re: [Qemu-devel] XSAVES in GET_SUPPORTED_CPUID (was Re: [PATCH] target-i386: add Skylake-Client cpu mode)

2016-05-03 Thread Dave Hansen
On 05/03/2016 11:23 AM, Eduardo Habkost wrote: > On Tue, May 03, 2016 at 10:53:43AM -0700, Dave Hansen wrote: >> On 05/03/2016 10:44 AM, Xiao Guangrong wrote: >>>>> This is the reason why setup_clear_cpu_cap(X86_FEATURE_XSAVES) >>>>> introduced by commit e882

Re: [PATCH v7 14/14] memfd_create.2: Describe MFD_INACCESSIBLE flag

2022-08-01 Thread Dave Hansen
This patch does not belong in this series. It's not a patch to the kernel. This is a kernel series. It would be much more appropriate to put a link to a separately posted manpage patch in the cover letter.

Re: [Qemu-devel] [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-10-26 Thread Dave Hansen
On 10/26/2016 03:06 AM, Li, Liang Z wrote: > I am working on Dave's new bitmap schema, I have finished the part of > getting the 'hybrid scheme bitmap' and found the complexity was more > than I expected. The main issue is more memory is required to save > the 'hybrid scheme bitmap' beside that use

Re: [Qemu-devel] [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-10-26 Thread Dave Hansen
On 10/26/2016 03:13 AM, Li, Liang Z wrote: > 3 times memory required is not accurate, please ignore this. sorry ... > The complexity is the point. What is making it so complex? Can you describe the problems?

Re: [Qemu-devel] [PATCH kernel v4 7/7] virtio-balloon: tell host vm's unused page info

2016-11-04 Thread Dave Hansen
Please squish this and patch 5 together. It makes no sense to separate them. > +static void send_unused_pages_info(struct virtio_balloon *vb, > + unsigned long req_id) > +{ > + struct scatterlist sg_in; > + unsigned long pfn = 0, bmap_len, pfn_limit, last_pfn,

Re: [Qemu-devel] [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page info

2016-11-30 Thread Dave Hansen
On 11/30/2016 12:43 AM, Liang Li wrote: > +static void send_unused_pages_info(struct virtio_balloon *vb, > + unsigned long req_id) > +{ > + struct scatterlist sg_in; > + unsigned long pos = 0; > + struct virtqueue *vq = vb->req_vq; > + struct virtio_ballo

Re: [Qemu-devel] [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page info

2016-12-05 Thread Dave Hansen
On 12/04/2016 05:13 AM, Li, Liang Z wrote: >> On 11/30/2016 12:43 AM, Liang Li wrote: >>> +static void send_unused_pages_info(struct virtio_balloon *vb, >>> + unsigned long req_id) >>> +{ >>> + struct scatterlist sg_in; >>> + unsigned long pos = 0; >>> + struct virtq

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-07 Thread Dave Hansen
On 12/07/2016 05:35 AM, Li, Liang Z wrote: >> Am 30.11.2016 um 09:43 schrieb Liang Li: >> IOW in real examples, do we have really large consecutive areas or are all >> pages just completely distributed over our memory? > > The buddy system of Linux kernel memory management shows there should > be

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-07 Thread Dave Hansen
On 12/07/2016 07:42 AM, David Hildenbrand wrote: > Am 07.12.2016 um 14:35 schrieb Li, Liang Z: >>> Am 30.11.2016 um 09:43 schrieb Liang Li: This patch set contains two parts of changes to the virtio-balloon. One is the change for speeding up the inflating & deflating process, th

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-07 Thread Dave Hansen
Removing silly virtio-dev@ list because it's bouncing mail... On 12/07/2016 08:21 AM, David Hildenbrand wrote: >> Li's current patches do that. Well, maybe not pfn/length, but they do >> take a pfn and page-order, which fits perfectly with the kernel's >> concept of high-order pages. > > So we c

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-07 Thread Dave Hansen
On 12/07/2016 10:38 AM, Andrea Arcangeli wrote: >> > and leaves room for the bitmap size to be encoded as well, if we decide >> > we need a bitmap in the future. > How would a bitmap ever be useful with very large page-order? Please, guys. Read the patches. *Please*. The current code doesn't ev

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-07 Thread Dave Hansen
2016 10:38 AM, Andrea Arcangeli wrote: > On Wed, Dec 07, 2016 at 08:57:01AM -0800, Dave Hansen wrote: >> It is more space-efficient. We're fitting the order into 6 bits, which >> would allows the full 2^64 address space to be represented in one entry, > > Very large orde

Re: [Qemu-devel] [PATCH kernel v4 7/7] virtio-balloon: tell host vm's unused page info

2016-11-07 Thread Dave Hansen
On 11/06/2016 07:37 PM, Li, Liang Z wrote: >> Let's say we do a 32k bitmap that can hold ~1M pages. That's 4GB of RAM. >> On a 1TB system, that's 256 passes through the top-level loop. >> The bottom-level lists have tens of thousands of pages in them, even on my >> laptop. Only 1/256 of these pag

Re: [Qemu-devel] [PATCH kernel v4 7/7] virtio-balloon: tell host vm's unused page info

2016-11-08 Thread Dave Hansen
On 11/07/2016 09:50 PM, Li, Liang Z wrote: > Sounds good. Should we ignore some of the order-0 pages in step 4 if the > bitmap is full? > Or should retry to get a complete list of order-0 pages? I think that's a pretty reasonable thing to do. >>> It seems the benefit we get for this feature is

Re: [Qemu-devel] [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-10-21 Thread Dave Hansen
On 10/20/2016 11:24 PM, Liang Li wrote: > Dave Hansen suggested a new scheme to encode the data structure, > because of additional complexity, it's not implemented in v3. So, what do you want done with this patch set? Do you want it applied as-is so that we can introduce a new hos

Re: [Qemu-devel] [RESEND PATCH v3 kernel 3/7] mm: add a function to get the max pfn

2016-10-24 Thread Dave Hansen
On 10/20/2016 11:24 PM, Liang Li wrote: > Expose the function to get the max pfn, so it can be used in the > virtio-balloon device driver. Simply include the 'linux/bootmem.h' > is not enough, if the device driver is built to a module, directly > refer the max_pfn lead to build failed. I'm not sur

Re: [Qemu-devel] [RESEND PATCH v3 kernel 1/7] virtio-balloon: rework deflate to add page to a list

2016-10-24 Thread Dave Hansen
On 10/20/2016 11:24 PM, Liang Li wrote: > Will allow faster notifications using a bitmap down the road. > balloon_pfn_to_page() can be removed because it's useless. This is a pretty terse description of what's going on here. Could you try to elaborate a bit? What *is* the current approach? Why

Re: [Qemu-devel] [RESEND PATCH v3 kernel 2/7] virtio-balloon: define new feature bit and page bitmap head

2016-10-24 Thread Dave Hansen
On 10/20/2016 11:24 PM, Liang Li wrote: > Add a new feature which supports sending the page information with > a bitmap. The current implementation uses PFNs array, which is not > very efficient. Using bitmap can improve the performance of > inflating/deflating significantly Why is it not efficien

[Qemu-devel] [PATCH] i386 translation: fix typo in xsetbv implementation

2016-07-20 Thread Dave Hansen
here else in the file when folks want to end the translation buffer. Richard Henderson confirmed that this is a bug and that this is the correct fix. Signed-off-by: Dave Hansen Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Richard Henderson --- target-i386/translate.c | 2 +- 1 file changed, 1

Re: [Qemu-devel] [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Dave Hansen
On 07/26/2016 06:23 PM, Liang Li wrote: > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; > + vb->pfn_limit = min(vb->pfn_limit, get_max_pfn()); > + vb->bmap_len = ALIGN(vb->pfn_limit, BITS_PER_LONG) / > + BITS_PER_BYTE + 2 * sizeof(unsigned long); > + hdr_len = sizeof(str

Re: [Qemu-devel] [PATCH v2 repost 6/7] mm: add the related functions to get free page info

2016-07-27 Thread Dave Hansen
On 07/26/2016 06:23 PM, Liang Li wrote: > + for_each_migratetype_order(order, t) { > + list_for_each(curr, &zone->free_area[order].free_list[t]) { > + pfn = page_to_pfn(list_entry(curr, struct page, lru)); > + if (pfn >= start_pfn && pfn <= en

Re: [Qemu-devel] [PATCH v2 repost 6/7] mm: add the related functions to get free page info

2016-07-27 Thread Dave Hansen
On 07/27/2016 03:05 PM, Michael S. Tsirkin wrote: > On Wed, Jul 27, 2016 at 09:40:56AM -0700, Dave Hansen wrote: >> On 07/26/2016 06:23 PM, Liang Li wrote: >>> + for_each_migratetype_order(order, t) { >>> + list_for_each(curr, &zon

Re: [Qemu-devel] [PATCH v2 repost 3/7] mm: add a function to get the max pfn

2016-07-27 Thread Dave Hansen
On 07/27/2016 03:08 PM, Michael S. Tsirkin wrote: >> > +unsigned long get_max_pfn(void) >> > +{ >> > + return max_pfn; >> > +} >> > +EXPORT_SYMBOL(get_max_pfn); >> > + > > This needs a coment that this can change at any time. > So it's only good as a hint e.g. for sizing data structures. Or, if

Re: [Qemu-devel] [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-29 Thread Dave Hansen
On 07/28/2016 02:51 PM, Michael S. Tsirkin wrote: >> > If 1MB is too big, how about 512K, or 256K? 32K seems too small. >> > > It's only small because it makes you rescan the free list. > So maybe you should do something else. > I looked at it a bit. Instead of scanning the free list, how about >

Re: [Qemu-devel] [PATCH v11 6/6] virtio-balloon: VIRTIO_BALLOON_F_CMD_VQ

2017-06-12 Thread Dave Hansen
On 06/09/2017 03:41 AM, Wei Wang wrote: > + for_each_populated_zone(zone) { > + for (order = MAX_ORDER - 1; order > 0; order--) { > + for (migratetype = 0; migratetype < MIGRATE_TYPES; > + migratetype++) { > +

Re: [Qemu-devel] [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-12 Thread Dave Hansen
Please stop cc'ing me on things also sent to closed mailing lists (virtio-...@lists.oasis-open.org). I'm happy to review things on open lists, but I'm not fond of the closed lists bouncing things at me. On 06/09/2017 03:41 AM, Wei Wang wrote: > Add a function to find a page block on the free list

Re: [Qemu-devel] [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-12 Thread Dave Hansen
On 06/12/2017 09:28 AM, Michael S. Tsirkin wrote: > >> The hypervisor is going to throw away the contents of these pages, >> right? > It should be careful and only throw away contents that was there before > report_unused_page_block was invoked. Hypervisor is responsible for not > corrupting gues

Re: [Qemu-devel] [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-12 Thread Dave Hansen
On 06/12/2017 01:34 PM, Michael S. Tsirkin wrote: > On Mon, Jun 12, 2017 at 09:42:36AM -0700, Dave Hansen wrote: >> On 06/12/2017 09:28 AM, Michael S. Tsirkin wrote: >>> >>>> The hypervisor is going to throw away the contents of these pages, >>>> right? &g

Re: [Qemu-devel] [PATCH v11 3/6] virtio-balloon: VIRTIO_BALLOON_F_PAGE_CHUNKS

2017-06-13 Thread Dave Hansen
On 06/13/2017 10:56 AM, Michael S. Tsirkin wrote: >> +/* The size of one page_bmap used to record inflated/deflated pages. */ >> +#define VIRTIO_BALLOON_PAGE_BMAP_SIZE (8 * PAGE_SIZE) > At this size, you probably want alloc_pages to avoid kmalloc > overhead. For slub, at least, kmalloc() jus

Re: [Qemu-devel] [PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-08-08 Thread Dave Hansen
On 08/07/2016 11:35 PM, Liang Li wrote: > Dave Hansen suggested a new scheme to encode the data structure, > because of additional complexity, it's not implemented in v3. FWIW, I don't think it takes any additional complexity here, at least in the guest implementation side. The t

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-08 Thread Dave Hansen
On 12/08/2016 08:45 PM, Li, Liang Z wrote: > What's the conclusion of your discussion? It seems you want some > statistic before deciding whether to ripping the bitmap from the > ABI, am I right? I think Andrea and David feel pretty strongly that we should remove the bitmap, unless we have some d

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-15 Thread Dave Hansen
On 12/14/2016 12:59 AM, Li, Liang Z wrote: >> Subject: Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for >> fast (de)inflating & fast live migration >> >> On 12/08/2016 08:45 PM, Li, Liang Z wrote: >>> What's the conclusion of your discussion? It seems you want some >>> statistic bef

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-15 Thread Dave Hansen
On 12/15/2016 04:48 PM, Li, Liang Z wrote: >>> It seems we leave too many bit for the pfn, and the bits leave for >>> length is not enough, How about keep 45 bits for the pfn and 19 bits >>> for length, 45 bits for pfn can cover 57 bits physical address, that should >>> be >> enough in the near f

Re: [Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-15 Thread Dave Hansen
On 12/15/2016 05:38 PM, Li, Liang Z wrote: > > Use 52 bits for 'pfn', 12 bits for 'length', when the 12 bits is not long > enough for the 'length' > Set the 'length' to a special value to indicate the "actual length in next 8 > bytes". > > That will be much more simple. Right? Sounds fine to m