Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-26 Thread Michal Hocko
[Sorry to jump in that late] On Tue 20-11-12 10:02:45, David Rientjes wrote: > On Mon, 19 Nov 2012, Glauber Costa wrote: > > > >> In the case I outlined below, for backwards compatibility. What I > > >> actually mean is that memcg *currently* allows arbitrary notifications. > > >> One way to merg

RE: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-21 Thread leonid.moiseichuk
-Original Message- From: ext Kirill A. Shutemov [mailto:kir...@shutemov.name] Sent: 21 November, 2012 11:31 ... BTW, there's interface for OOM notification in memcg. See oom_control. I guess other pressure levels can also fit to the interface. --- Hi, I have tracking this conversation v

RE: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-21 Thread leonid.moiseichuk
-Original Message- From: ext Glauber Costa [mailto:glom...@parallels.com] Sent: 21 November, 2012 13:55 So I'll say it again: if this is always global, there is no reason any cgroup needs to be involved. If this turns out to be per-process, as Anton suggested in a recent e-mail, I do

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-21 Thread Glauber Costa
Hi, > > Memory notifications are quite irrelevant to partitioning and cgroups. The > use-case is related to user-space handling low memory. Meaning the > functionality should be accurate with specific granularity (e.g. 1 MB) and > time (0.25s is OK) but better to have it as simple and battery-fr

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-21 Thread Kirill A. Shutemov
On Tue, Nov 20, 2012 at 10:02:45AM -0800, David Rientjes wrote: > On Mon, 19 Nov 2012, Glauber Costa wrote: > > > >> In the case I outlined below, for backwards compatibility. What I > > >> actually mean is that memcg *currently* allows arbitrary notifications. > > >> One way to merge those, while

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-21 Thread Glauber Costa
On 11/21/2012 12:46 PM, Anton Vorontsov wrote: > On Wed, Nov 21, 2012 at 12:27:28PM +0400, Glauber Costa wrote: >> On 11/20/2012 10:23 PM, David Rientjes wrote: >>> Anton can correct me if I'm wrong, but I certainly don't think this is >>> where mempressure is headed: I don't think any accounting

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-21 Thread Anton Vorontsov
On Wed, Nov 21, 2012 at 12:27:28PM +0400, Glauber Costa wrote: > On 11/20/2012 10:23 PM, David Rientjes wrote: > > Anton can correct me if I'm wrong, but I certainly don't think this is > > where mempressure is headed: I don't think any accounting needs to be done Yup, I'd rather not do any accou

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-21 Thread Glauber Costa
On 11/20/2012 10:23 PM, David Rientjes wrote: > Anton can correct me if I'm wrong, but I certainly don't think this is > where mempressure is headed: I don't think any accounting needs to be done > and, if it is, it's a design issue that should be addressed now rather > than later. I believe no

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-20 Thread David Rientjes
On Mon, 19 Nov 2012, Glauber Costa wrote: > >> Because cpusets only deal with memory placement, not memory usage. > > > > The set of nodes that a thread is allowed to allocate from may face memory > > pressure up to and including oom while the rest of the system may have a > > ton of free memor

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-20 Thread David Rientjes
On Mon, 19 Nov 2012, Glauber Costa wrote: > >> In the case I outlined below, for backwards compatibility. What I > >> actually mean is that memcg *currently* allows arbitrary notifications. > >> One way to merge those, while moving to a saner 3-point notification, is > >> to still allow the old wr

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-19 Thread Glauber Costa
>>> Umm, why do users of cpusets not want to be able to trigger memory >>> pressure notifications? >>> >> Because cpusets only deal with memory placement, not memory usage. > > The set of nodes that a thread is allowed to allocate from may face memory > pressure up to and including oom while th

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-19 Thread Glauber Costa
On 11/17/2012 05:21 AM, Anton Vorontsov wrote: > On Fri, Nov 16, 2012 at 01:57:09PM -0800, David Rientjes wrote: I'm wondering if we should have more than three different levels. >>> >>> In the case I outlined below, for backwards compatibility. What I >>> actually mean is that memcg *cur

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-19 Thread Glauber Costa
On 11/17/2012 01:57 AM, David Rientjes wrote: > On Sat, 17 Nov 2012, Glauber Costa wrote: > >>> I'm wondering if we should have more than three different levels. >>> >> >> In the case I outlined below, for backwards compatibility. What I >> actually mean is that memcg *currently* allows arbitrary

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-18 Thread David Rientjes
On Fri, 16 Nov 2012, Anton Vorontsov wrote: > The main change is that I decided to go with discrete levels of the > pressure. > > When I started writing the man page, I had to describe the 'reclaimer > inefficiency index', and while doing this I realized that I'm describing > how the kernel

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-16 Thread Anton Vorontsov
On Fri, Nov 16, 2012 at 01:57:09PM -0800, David Rientjes wrote: > > > I'm wondering if we should have more than three different levels. > > > > > > > In the case I outlined below, for backwards compatibility. What I > > actually mean is that memcg *currently* allows arbitrary notifications. > > O

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-16 Thread David Rientjes
On Sat, 17 Nov 2012, Glauber Costa wrote: > > I'm wondering if we should have more than three different levels. > > > > In the case I outlined below, for backwards compatibility. What I > actually mean is that memcg *currently* allows arbitrary notifications. > One way to merge those, while movi

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-16 Thread Glauber Costa
Hey, On 11/17/2012 12:04 AM, David Rientjes wrote: > On Fri, 16 Nov 2012, Glauber Costa wrote: > >> My personal take: >> >> Most people hate memcg due to the cost it imposes. I've already >> demonstrated that with some effort, it doesn't necessarily have to be >> so. (http://lwn.net/Articles/517

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-16 Thread David Rientjes
On Fri, 16 Nov 2012, Glauber Costa wrote: > My personal take: > > Most people hate memcg due to the cost it imposes. I've already > demonstrated that with some effort, it doesn't necessarily have to be > so. (http://lwn.net/Articles/517634/) > > The one thing I missed on that work, was precisely

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-16 Thread Glauber Costa
On 11/16/2012 01:25 AM, David Rientjes wrote: > On Thu, 15 Nov 2012, Anton Vorontsov wrote: > >> Hehe, you're saying that we have to have cgroups=y. :) But some folks were >> deliberately asking us to make the cgroups optional. >> > > Enabling just CONFIG_CGROUPS (which is enabled by default) and

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-15 Thread David Rientjes
On Thu, 15 Nov 2012, Anton Vorontsov wrote: > Hehe, you're saying that we have to have cgroups=y. :) But some folks were > deliberately asking us to make the cgroups optional. > Enabling just CONFIG_CGROUPS (which is enabled by default) and no other current cgroups increases the size of the ker

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-15 Thread Anton Vorontsov
On Thu, Nov 15, 2012 at 12:11:47AM -0800, David Rientjes wrote: [...] > Might not be too difficult if you implement your own cgroup to aggregate > these tasks for which you want to know memory pressure events; it would > have to be triggered for the task trying to allocate memory at any given >

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-15 Thread David Rientjes
On Wed, 14 Nov 2012, Anton Vorontsov wrote: > Thanks again for your inspirational comments! > Heh, not sure I've been too inspirational (probably more annoying than anything else). I really do want generic memory pressure notifications in the kernel and already have some ideas on how I can ti

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-14 Thread Anton Vorontsov
Hi David, Thanks again for your inspirational comments! On Wed, Nov 14, 2012 at 07:59:52PM -0800, David Rientjes wrote: > > > I agree that eventfd is the way to go, but I'll also add that this > > > feature > > > seems to be implemented at a far too coarse of level. Memory, and hence > > > me

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-14 Thread David Rientjes
On Wed, 14 Nov 2012, Anton Vorontsov wrote: > > I agree that eventfd is the way to go, but I'll also add that this feature > > seems to be implemented at a far too coarse of level. Memory, and hence > > memory pressure, is constrained by several factors other than just the > > amount of physic

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-14 Thread Anton Vorontsov
Hi David, Thanks for your comments! On Wed, Nov 14, 2012 at 07:21:14PM -0800, David Rientjes wrote: > > > Why should you be required to use cgroups to get VM pressure events to > > > userspace? > > > > Valid point. But in fact you have it on most systems anyway. > > > > I personally don't like

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-14 Thread David Rientjes
On Wed, 7 Nov 2012, Kirill A. Shutemov wrote: > > > Sorry, I didn't follow previous discussion on this, but could you > > > explain what's wrong with memory notifications from memcg? > > > As I can see you can get pretty similar functionality using memory > > > thresholds on the root cgroup. What'

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-09 Thread Anton Vorontsov
On Fri, Nov 09, 2012 at 09:32:03AM +0100, Luiz Capitulino wrote: > Anton Vorontsov wrote: > > This is the third RFC. As suggested by Minchan Kim, the API is much > > simplified now (comparing to vmevent_fd): > > Which tree is this against? I'd like to try this series, but it doesn't > apply to Li

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-09 Thread Luiz Capitulino
Hi Anton, On Wed, 7 Nov 2012 02:53:49 -0800 Anton Vorontsov wrote: > Hi all, > > This is the third RFC. As suggested by Minchan Kim, the API is much > simplified now (comparing to vmevent_fd): Which tree is this against? I'd like to try this series, but it doesn't apply to Linus tree. -- To un

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Pekka Enberg
Hi Greg, On 11/7/12 7:20 PM, Greg Thelen wrote: > Related question: are there plans to extend this system call to > provide per-cgroup vm pressure notification? Yes, that's something that needs to be addressed before we can ever consider merging something like this to mainline. We probably need

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Greg Thelen
On Wed, Nov 07 2012, Kirill A. Shutemov wrote: > On Wed, Nov 07, 2012 at 02:53:49AM -0800, Anton Vorontsov wrote: >> Hi all, >> >> This is the third RFC. As suggested by Minchan Kim, the API is much >> simplified now (comparing to vmevent_fd): >> >> - As well as Minchan, KOSAKI Motohiro didn't l

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Anton Vorontsov
On Wed, Nov 07, 2012 at 02:11:10PM +0200, Kirill A. Shutemov wrote: [...] > >We can have plenty of "free" memory, of which say 90% will be caches, > >and say 10% idle. But we do want to differentiate these types of memory > >(although not going into details about it), i.e. we want to ge

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Anton Vorontsov
On Wed, Nov 07, 2012 at 01:30:16PM +0200, Pekka Enberg wrote: [...] > I love the API and implementation simplifications but I hate the new > ABI. It's a specialized, single-purpose syscall and bunch of procfs > tunables and I don't see how it's 'extensible' to anything but VM It is extensible to V

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Kirill A. Shutemov
On Wed, Nov 07, 2012 at 03:43:46AM -0800, Anton Vorontsov wrote: > On Wed, Nov 07, 2012 at 01:21:36PM +0200, Kirill A. Shutemov wrote: > [...] > > Sorry, I didn't follow previous discussion on this, but could you > > explain what's wrong with memory notifications from memcg? > > As I can see you ca

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Anton Vorontsov
On Wed, Nov 07, 2012 at 01:21:36PM +0200, Kirill A. Shutemov wrote: [...] > Sorry, I didn't follow previous discussion on this, but could you > explain what's wrong with memory notifications from memcg? > As I can see you can get pretty similar functionality using memory > thresholds on the root cg

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Kirill A. Shutemov
On Wed, Nov 07, 2012 at 01:28:12PM +0200, Pekka Enberg wrote: > On Wed, Nov 7, 2012 at 1:21 PM, Kirill A. Shutemov > wrote: > >> While the new API is very simple, it is still extensible (i.e. versioned). > > > > Sorry, I didn't follow previous discussion on this, but could you > > explain what's

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Pekka Enberg
On Wed, Nov 7, 2012 at 1:30 PM, Pekka Enberg wrote: > I love the API and implementation simplifications but I hate the new > ABI. It's a specialized, single-purpose syscall and bunch of procfs > tunables and I don't see how it's 'extensible' to anything but VM s/anything but VM/anything but VM pr

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Pekka Enberg
Hi Anton, On Wed, Nov 7, 2012 at 12:53 PM, Anton Vorontsov wrote: > This is the third RFC. As suggested by Minchan Kim, the API is much > simplified now (comparing to vmevent_fd): > > - As well as Minchan, KOSAKI Motohiro didn't like the timers, so the > timers are gone now; > - Pekka Enberg di

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Pekka Enberg
On Wed, Nov 7, 2012 at 1:21 PM, Kirill A. Shutemov wrote: >> While the new API is very simple, it is still extensible (i.e. versioned). > > Sorry, I didn't follow previous discussion on this, but could you > explain what's wrong with memory notifications from memcg? > As I can see you can get pret

Re: [RFC v3 0/3] vmpressure_fd: Linux VM pressure notifications

2012-11-07 Thread Kirill A. Shutemov
On Wed, Nov 07, 2012 at 02:53:49AM -0800, Anton Vorontsov wrote: > Hi all, > > This is the third RFC. As suggested by Minchan Kim, the API is much > simplified now (comparing to vmevent_fd): > > - As well as Minchan, KOSAKI Motohiro didn't like the timers, so the > timers are gone now; > - Pekk