Re: Thoughts on simple scanner approach for free page hinting

2019-04-09 Thread Michael S. Tsirkin
On Tue, Apr 09, 2019 at 03:43:58PM +0200, David Hildenbrand wrote: > On 09.04.19 15:37, Michael S. Tsirkin wrote: > > On Tue, Apr 09, 2019 at 03:36:08PM +0200, David Hildenbrand wrote: > >> On 09.04.19 15:31, Michael S. Tsirkin wrote: > >>> On Tue, Apr 09, 2019 at 11:20:36AM +0200, David Hildenbran

Re: Thoughts on simple scanner approach for free page hinting

2019-04-09 Thread David Hildenbrand
On 09.04.19 15:37, Michael S. Tsirkin wrote: > On Tue, Apr 09, 2019 at 03:36:08PM +0200, David Hildenbrand wrote: >> On 09.04.19 15:31, Michael S. Tsirkin wrote: >>> On Tue, Apr 09, 2019 at 11:20:36AM +0200, David Hildenbrand wrote: BTW I like the idea of allocating pages that have already bee

Re: Thoughts on simple scanner approach for free page hinting

2019-04-09 Thread Michael S. Tsirkin
On Tue, Apr 09, 2019 at 03:36:08PM +0200, David Hildenbrand wrote: > On 09.04.19 15:31, Michael S. Tsirkin wrote: > > On Tue, Apr 09, 2019 at 11:20:36AM +0200, David Hildenbrand wrote: > >> BTW I like the idea of allocating pages that have already been hinted as > >> last "choice", allocating pages

Re: Thoughts on simple scanner approach for free page hinting

2019-04-09 Thread David Hildenbrand
On 09.04.19 15:31, Michael S. Tsirkin wrote: > On Tue, Apr 09, 2019 at 11:20:36AM +0200, David Hildenbrand wrote: >> BTW I like the idea of allocating pages that have already been hinted as >> last "choice", allocating pages that have not been hinted yet first. > > OK I guess but note this is just

Re: Thoughts on simple scanner approach for free page hinting

2019-04-09 Thread Michael S. Tsirkin
On Tue, Apr 09, 2019 at 11:20:36AM +0200, David Hildenbrand wrote: > BTW I like the idea of allocating pages that have already been hinted as > last "choice", allocating pages that have not been hinted yet first. OK I guess but note this is just a small window during which not all pages have been

Re: Thoughts on simple scanner approach for free page hinting

2019-04-09 Thread David Hildenbrand
> With that said I have a few ideas that may help to address the 4 > issues called out above. The basic idea is simple. We use a high water > mark based on zone->free_area[order].nr_free to determine when to wake > up a thread to start hinting memory out of a given free area. From > there we alloca

Re: Thoughts on simple scanner approach for free page hinting

2019-04-09 Thread David Hildenbrand
On 09.04.19 04:44, Michael S. Tsirkin wrote: > On Fri, Apr 05, 2019 at 05:09:45PM -0700, Alexander Duyck wrote: >> In addition we will need some way to identify which pages have been >> hinted on and which have not. The way I believe easiest to do this >> would be to overload the PageType value so

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Michael S. Tsirkin
On Fri, Apr 05, 2019 at 05:09:45PM -0700, Alexander Duyck wrote: > In addition we will need some way to identify which pages have been > hinted on and which have not. The way I believe easiest to do this > would be to overload the PageType value so that we could essentially > have two values for "B

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Alexander Duyck
On Mon, Apr 8, 2019 at 2:21 PM David Hildenbrand wrote: > > On 08.04.19 22:51, David Hildenbrand wrote: > > On 08.04.19 22:10, Alexander Duyck wrote: > >> On Mon, Apr 8, 2019 at 11:40 AM David Hildenbrand wrote: > >>> > >> > >> In addition we will need some way to identify which pages hav

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread David Hildenbrand
On 08.04.19 22:51, David Hildenbrand wrote: > On 08.04.19 22:10, Alexander Duyck wrote: >> On Mon, Apr 8, 2019 at 11:40 AM David Hildenbrand wrote: >>> >> >> In addition we will need some way to identify which pages have been >> hinted on and which have not. The way I believe easiest t

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread David Hildenbrand
On 08.04.19 22:10, Alexander Duyck wrote: > On Mon, Apr 8, 2019 at 11:40 AM David Hildenbrand wrote: >> > > In addition we will need some way to identify which pages have been > hinted on and which have not. The way I believe easiest to do this > would be to overload the PageType v

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread David Hildenbrand
On 08.04.19 22:10, Alexander Duyck wrote: > On Mon, Apr 8, 2019 at 11:40 AM David Hildenbrand wrote: >> > > In addition we will need some way to identify which pages have been > hinted on and which have not. The way I believe easiest to do this > would be to overload the PageType v

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Alexander Duyck
On Mon, Apr 8, 2019 at 11:40 AM David Hildenbrand wrote: > > >>> > >>> In addition we will need some way to identify which pages have been > >>> hinted on and which have not. The way I believe easiest to do this > >>> would be to overload the PageType value so that we could essentially > >>> have

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread David Hildenbrand
>>> - Define and use counters corresponding to each zone to monitor the >>> amount of memory freed. >>> - As soon as the 64MB free memory threshold is hit wake up the kernel >>> thread which will scan this xbitmap and try to isolate the pages and >>> clear the corresponding bits. (We still have to

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread David Hildenbrand
>>> >>> In addition we will need some way to identify which pages have been >>> hinted on and which have not. The way I believe easiest to do this >>> would be to overload the PageType value so that we could essentially >>> have two values for "Buddy" pages. We would have our standard "Buddy" >>> p

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Alexander Duyck
On Mon, Apr 8, 2019 at 11:19 AM Michael S. Tsirkin wrote: > > On Mon, Apr 08, 2019 at 02:09:59PM -0400, Nitesh Narayan Lal wrote: > > On 4/8/19 12:36 PM, David Hildenbrand wrote: > > > On 06.04.19 02:09, Alexander Duyck wrote: > > >> So I am starting this thread as a spot to collect my thoughts on

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread David Hildenbrand
On 08.04.19 20:09, Nitesh Narayan Lal wrote: > On 4/8/19 12:36 PM, David Hildenbrand wrote: >> On 06.04.19 02:09, Alexander Duyck wrote: >>> So I am starting this thread as a spot to collect my thoughts on the >>> current guest free page hinting design as well as point out a few >>> possible things

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Michael S. Tsirkin
On Mon, Apr 08, 2019 at 02:09:59PM -0400, Nitesh Narayan Lal wrote: > On 4/8/19 12:36 PM, David Hildenbrand wrote: > > On 06.04.19 02:09, Alexander Duyck wrote: > >> So I am starting this thread as a spot to collect my thoughts on the > >> current guest free page hinting design as well as point out

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Alexander Duyck
On Mon, Apr 8, 2019 at 9:36 AM David Hildenbrand wrote: > > On 06.04.19 02:09, Alexander Duyck wrote: > > So I am starting this thread as a spot to collect my thoughts on the > > current guest free page hinting design as well as point out a few > > possible things we could do to improve upon it. >

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Nitesh Narayan Lal
On 4/8/19 12:36 PM, David Hildenbrand wrote: > On 06.04.19 02:09, Alexander Duyck wrote: >> So I am starting this thread as a spot to collect my thoughts on the >> current guest free page hinting design as well as point out a few >> possible things we could do to improve upon it. >> >> 1. The curre

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread David Hildenbrand
On 06.04.19 02:09, Alexander Duyck wrote: > So I am starting this thread as a spot to collect my thoughts on the > current guest free page hinting design as well as point out a few > possible things we could do to improve upon it. > > 1. The current design isn't likely going to scale well to multi

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Michael S. Tsirkin
On Mon, Apr 08, 2019 at 08:18:35AM -0700, Alexander Duyck wrote: > On Mon, Apr 8, 2019 at 5:24 AM Nitesh Narayan Lal wrote: > > > > > > On 4/5/19 8:09 PM, Alexander Duyck wrote: > > > So I am starting this thread as a spot to collect my thoughts on the > > > current guest free page hinting design

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Alexander Duyck
On Mon, Apr 8, 2019 at 5:24 AM Nitesh Narayan Lal wrote: > > > On 4/5/19 8:09 PM, Alexander Duyck wrote: > > So I am starting this thread as a spot to collect my thoughts on the > > current guest free page hinting design as well as point out a few > > possible things we could do to improve upon it

Re: Thoughts on simple scanner approach for free page hinting

2019-04-08 Thread Nitesh Narayan Lal
On 4/5/19 8:09 PM, Alexander Duyck wrote: > So I am starting this thread as a spot to collect my thoughts on the > current guest free page hinting design as well as point out a few > possible things we could do to improve upon it. > > 1. The current design isn't likely going to scale well to multi

Thoughts on simple scanner approach for free page hinting

2019-04-05 Thread Alexander Duyck
So I am starting this thread as a spot to collect my thoughts on the current guest free page hinting design as well as point out a few possible things we could do to improve upon it. 1. The current design isn't likely going to scale well to multiple VCPUs. The issue specifically is that the zone l