Re: [v4 2/4] mm, oom: cgroup-aware OOM killer

2017-08-01 Thread Michal Hocko
On Tue 01-08-17 16:25:48, Roman Gushchin wrote: > On Tue, Aug 01, 2017 at 04:54:35PM +0200, Michal Hocko wrote: [...] > > I would reap out the oom_kill_process into a separate patch. > > It was a separate patch, I've merged it based on Vladimir's feedback. > No problem

Re: [v4 2/4] mm, oom: cgroup-aware OOM killer

2017-08-02 Thread Michal Hocko
On Tue 01-08-17 19:13:52, Roman Gushchin wrote: > On Tue, Aug 01, 2017 at 07:03:03PM +0200, Michal Hocko wrote: > > On Tue 01-08-17 16:25:48, Roman Gushchin wrote: > > > On Tue, Aug 01, 2017 at 04:54:35PM +0200, Michal Hocko wrote: > > [...] > > > > I would

Re: [v4 2/4] mm, oom: cgroup-aware OOM killer

2017-08-03 Thread Michal Hocko
On Thu 03-08-17 13:47:51, Roman Gushchin wrote: > On Wed, Aug 02, 2017 at 09:29:01AM +0200, Michal Hocko wrote: > > On Tue 01-08-17 19:13:52, Roman Gushchin wrote: > > > On Tue, Aug 01, 2017 at 07:03:03PM +0200, Michal Hocko wrote: > > > > On Tue 01-08-17

Re: [v6 1/4] mm, oom: refactor the oom_kill_process() function

2017-08-24 Thread Michal Hocko
task, as well as play > with task selection (considering task's children), > so we can't use the existing oom_kill_process(). > > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc: Vladimir Davydov > Cc: Johannes Weiner > Cc: Tetsuo Handa > Cc: David Rientjes >

Re: [v6 2/4] mm, oom: cgroup-aware OOM killer

2017-08-24 Thread Michal Hocko
re going any further. I believe that users shouldn't see any difference in the OOM behavior when memcg v2 is used and there is no kill-all memcg. If there is such a memcg then we should treat only those specially. But you might have really strong usecases which haven't been presented or I've missed their importance. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v6 3/4] mm, oom: introduce oom_priority for memory cgroups

2017-08-24 Thread Michal Hocko
} Just a minor thing. Why do we even have to calculate oom_score when we use it only as a tiebreaker? -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v6 2/4] mm, oom: cgroup-aware OOM killer

2017-08-24 Thread Michal Hocko
On Thu 24-08-17 13:28:46, Roman Gushchin wrote: > Hi Michal! > > On Thu, Aug 24, 2017 at 01:47:06PM +0200, Michal Hocko wrote: > > This doesn't apply on top of mmotm cleanly. You are missing > > http://lkml.kernel.org/r/20170807113839.16695-3-mho...@kernel.or

Re: [v6 3/4] mm, oom: introduce oom_priority for memory cgroups

2017-08-24 Thread Michal Hocko
On Thu 24-08-17 13:51:13, Roman Gushchin wrote: > On Thu, Aug 24, 2017 at 02:10:54PM +0200, Michal Hocko wrote: > > On Wed 23-08-17 17:52:00, Roman Gushchin wrote: > > > Introduce a per-memory-cgroup oom_priority setting: an integer number > > > within the [-1, 1000

Re: [v6 2/4] mm, oom: cgroup-aware OOM killer

2017-08-24 Thread Michal Hocko
On Thu 24-08-17 14:58:42, Roman Gushchin wrote: > On Thu, Aug 24, 2017 at 02:58:11PM +0200, Michal Hocko wrote: > > On Thu 24-08-17 13:28:46, Roman Gushchin wrote: > > > Hi Michal! > > > > > There is nothing like a "better victim". We are pretty much in

Re: [v6 2/4] mm, oom: cgroup-aware OOM killer

2017-08-25 Thread Michal Hocko
On Thu 24-08-17 15:58:01, Roman Gushchin wrote: > On Thu, Aug 24, 2017 at 04:13:37PM +0200, Michal Hocko wrote: > > On Thu 24-08-17 14:58:42, Roman Gushchin wrote: [...] > > > Both ways are not ideal, and sum of the processes is not ideal too. > > > Especially, if

Re: [v6 2/4] mm, oom: cgroup-aware OOM killer

2017-08-25 Thread Michal Hocko
On Fri 25-08-17 11:39:51, Roman Gushchin wrote: > On Fri, Aug 25, 2017 at 10:14:03AM +0200, Michal Hocko wrote: > > On Thu 24-08-17 15:58:01, Roman Gushchin wrote: > > > On Thu, Aug 24, 2017 at 04:13:37PM +0200, Michal Hocko wrote: > > > > On Thu 24-08-17

Re: [v7 1/5] mm, oom: refactor the oom_kill_process() function

2017-09-05 Thread Michal Hocko
m cgroup. We don't need to print > the debug information for the each task, as well as play > with task selection (considering task's children), > so we can't use the existing oom_kill_process(). > > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-05 Thread Michal Hocko
iority and kill-all are a natural extension of this new strategy. This will make the life easier and easier to understand by users. Does that make sense to you? > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc: Vladimir Davydov > Cc: Johannes Weiner > Cc: Tetsuo Handa

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-05 Thread Michal Hocko
ex than necessary. With "tasks only in leafs" cgroup policy we should only see any pages on LRUs on the global root memcg and leaf cgroups. The same applies to memcg stats. So why cannot we simply do the tree walk, calculate badness/check the priority and select the largest memcg in one go?

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-05 Thread Michal Hocko
On Tue 05-09-17 15:30:21, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 03:44:12PM +0200, Michal Hocko wrote: [...] > > Why is this an opt out rather than opt-in? IMHO the original oom logic > > should be preserved by default and specific workloads should opt in for > > t

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 17:53:44, Johannes Weiner wrote: > On Tue, Sep 05, 2017 at 03:44:12PM +0200, Michal Hocko wrote: > > Why is this an opt out rather than opt-in? IMHO the original oom logic > > should be preserved by default and specific workloads should opt in for > > t

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 21:23:57, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 04:57:00PM +0200, Michal Hocko wrote: [...] > > > @@ -810,6 +810,9 @@ static void __oom_kill_process(struct task_struct > > > *victim) > > > struct mm_struct *mm; > > > bool

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 21:23:57, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 04:57:00PM +0200, Michal Hocko wrote: [...] > > Hmm. The changelog says "By default, it will look for the biggest leaf > > cgroup, and kill the largest task inside." But you are accumulating >

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Tue 05-09-17 20:16:09, Roman Gushchin wrote: > On Tue, Sep 05, 2017 at 05:12:51PM +0200, Michal Hocko wrote: [...] > > > Then we should probably hide corresponding > > > cgroup interface (oom_group and oom_priority knobs) by default, > > > and it feels as unnecessa

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
On Wed 06-09-17 13:57:50, Roman Gushchin wrote: > On Wed, Sep 06, 2017 at 10:31:58AM +0200, Michal Hocko wrote: > > On Tue 05-09-17 21:23:57, Roman Gushchin wrote: > > > On Tue, Sep 05, 2017 at 04:57:00PM +0200, Michal Hocko wrote: > > [...] > > > > Hmm. The

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
experience. There are special cases of course but we should target general use first. Kill the whole memcg is a really useful feature on its own for proper container cleanup. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the bo

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-06 Thread Michal Hocko
o completely deprecate it at some point. > > To make a first step towards deprecation, let's warn potential > users about deprecation plans. > > Signed-off-by: Roman Gushchin > Cc: Andrew Morton > Cc: Michal Hocko > Cc: Johannes Weiner > Cc: David Rie

Re: [v7 2/5] mm, oom: cgroup-aware OOM killer

2017-09-11 Thread Michal Hocko
dness does for the regular oom killer. Or do you have anything else in mind? -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v7 5/5] mm, oom: cgroup v2 mount option to disable cgroup-aware OOM killer

2017-09-11 Thread Michal Hocko
On Thu 07-09-17 12:14:57, Johannes Weiner wrote: > On Wed, Sep 06, 2017 at 10:28:59AM +0200, Michal Hocko wrote: > > On Tue 05-09-17 17:53:44, Johannes Weiner wrote: > > > The cgroup-awareness in the OOM killer is exactly the same thing. It > > > should have been the

Re: [v8 3/4] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-09-13 Thread Michal Hocko
> So, the only question is should it be opt-in or opt-out option. > Personally, I would prefer opt-out, but Michal has a very strong opinion here. Yes I still strongly believe this has to be opt-in. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-13 Thread Michal Hocko
memcgs is more straightforward and it doesn't lead to unexpected results as mentioned before (kill a small memcg which is a part of the larger sub-hierarchy). I didn't get to read the new version of this series yet and hope to get to it soon. -- Michal Hocko SUSE Labs -- To unsubscribe f

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-14 Thread Michal Hocko
On Wed 13-09-17 13:46:08, David Rientjes wrote: > On Wed, 13 Sep 2017, Michal Hocko wrote: > > > > > This patchset makes the OOM killer cgroup-aware. > > > > > > > > v8: > > > > - Do not kill tasks with OOM_SCORE_ADJ -1000 > >

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-14 Thread Michal Hocko
On Wed 13-09-17 14:56:07, Roman Gushchin wrote: > On Wed, Sep 13, 2017 at 02:29:14PM +0200, Michal Hocko wrote: [...] > > I strongly believe that comparing only leaf memcgs > > is more straightforward and it doesn't lead to unexpected results as > > mentioned before (kil

Re: [v8 1/4] mm, oom: refactor the oom_kill_process() function

2017-09-14 Thread Michal Hocko
m cgroup. We don't need to print > the debug information for the each task, as well as play > with task selection (considering task's children), > so we can't use the existing oom_kill_process(). > > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-15 Thread Michal Hocko
On Thu 14-09-17 09:05:48, Roman Gushchin wrote: > On Thu, Sep 14, 2017 at 03:40:14PM +0200, Michal Hocko wrote: > > On Wed 13-09-17 14:56:07, Roman Gushchin wrote: > > > On Wed, Sep 13, 2017 at 02:29:14PM +0200, Michal Hocko wrote: > > [...] > > > > I stron

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-17 Thread Michal Hocko
On Fri 15-09-17 08:23:01, Roman Gushchin wrote: > On Fri, Sep 15, 2017 at 12:58:26PM +0200, Michal Hocko wrote: > > On Thu 14-09-17 09:05:48, Roman Gushchin wrote: > > > On Thu, Sep 14, 2017 at 03:40:14PM +0200, Michal Hocko wrote: > > > > On Wed 13-09-17

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-17 Thread Michal Hocko
t; cgroups and they can own memory.oom_priority for their own subcontainers, > this becomes quite powerful so they can define their own oom priorities. > Otherwise, they can easily override the oom priorities of other cgroups. Could you be more speicific about your usecase? What would be

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-17 Thread Michal Hocko
mpare memcgs, which are on different levels > (or in different subtrees). Well, I have given you one that doesn't sounds completely insane to me in other email. You may need an intermediate level for other than memcg controller. The whole concept of significance of the hierarchy level seems really odd to me. Or am I wrong here? -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-25 Thread Michal Hocko
I would really appreciate some feedback from Tejun, Johannes here. On Wed 20-09-17 14:53:41, Roman Gushchin wrote: > On Mon, Sep 18, 2017 at 08:14:05AM +0200, Michal Hocko wrote: > > On Fri 15-09-17 08:23:01, Roman Gushchin wrote: > > > On Fri, Sep 15, 2017 at 12:58:26PM +0200,

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-25 Thread Michal Hocko
e killed. An obvious example will > be a task with oom_score_adj set to any non-extreme (other than 0 and -1000) > value, but it can also happen in case of constrained alloc, for instance. I am not sure I understand. Are you talking about root memcg comparing to other memcgs? -- Michal Hoc

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-26 Thread Michal Hocko
tricky and I would even dare to claim a wrong semantic. I can see priorities being very useful on killable entities for sure. I am not entirely sure what would be the best approach yet and that is why I've suggested that to postpone to after we settle with a simple approach first. Bringing pr

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-26 Thread Michal Hocko
On Tue 26-09-17 11:59:25, Roman Gushchin wrote: > On Mon, Sep 25, 2017 at 10:25:21PM +0200, Michal Hocko wrote: > > On Mon 25-09-17 19:15:33, Roman Gushchin wrote: > > [...] > > > I'm not against this model, as I've said before. It feels logical, >

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-26 Thread Michal Hocko
On Tue 26-09-17 13:13:00, Roman Gushchin wrote: > On Tue, Sep 26, 2017 at 01:21:34PM +0200, Michal Hocko wrote: > > On Tue 26-09-17 11:59:25, Roman Gushchin wrote: > > > On Mon, Sep 25, 2017 at 10:25:21PM +0200, Michal Hocko wrote: > > > > On Mon 25-09-17

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-27 Thread Michal Hocko
On Tue 26-09-17 14:04:41, David Rientjes wrote: > On Tue, 26 Sep 2017, Michal Hocko wrote: > > > > No, I agree that we shouldn't compare sibling memory cgroups based on > > > different criteria depending on whether group_oom is set or not. > > > >

Re: [v8 0/4] cgroup-aware OOM killer

2017-09-27 Thread Michal Hocko
er argument for that example yet. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
ority based, BFP based or module based selection. But let's start simple with the most basic scenario first with a most sensible semantic implemented. I believe the latest version (v9) looks sensible from the semantic point of view and we should focus on making it into a mergeable shape. --

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-02 Thread Michal Hocko
lb pages are not really migratable which should be the only criterion. Hugetlb pages are no different from other migratable pages in that regards. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 16:06:33, Alexandru Moise wrote: > On Mon, Oct 02, 2017 at 02:54:32PM +0200, Michal Hocko wrote: > > On Mon 02-10-17 00:51:11, Alexandru Moise wrote: > > > This attempts to bring more flexibility to how hugepages are allocated > > > by making it possibl

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 13:47:12, Roman Gushchin wrote: > On Mon, Oct 02, 2017 at 02:24:34PM +0200, Michal Hocko wrote: [...] > > I believe the latest version (v9) looks sensible from the semantic point > > of view and we should focus on making it into a mergeable shape. > > The onl

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 17:06:38, Alexandru Moise wrote: > On Mon, Oct 02, 2017 at 04:27:17PM +0200, Michal Hocko wrote: > > On Mon 02-10-17 16:06:33, Alexandru Moise wrote: > > > On Mon, Oct 02, 2017 at 02:54:32PM +0200, Michal Hocko wrote: > > > > On Mon 02-10-17 0

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
orry but I would really appreciate to focus on making the step 1 done before diverging into details about potential improvements and a better control over the selection. This whole thing is an opt-in so there is a no risk of a regression. -- Michal Hocko SUSE Labs -- To unsubscribe from this l

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
semantics from it. oom_group _is_ about killall semantic. And comparing killable entities is just a natural thing to do. So I am not sure what you mean -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to major

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
understand the proposal (from > reading thread, not patch) it does not. No it doesn't. It allows you to kill A (recursively) as the largest memory consumer. So, no, it cannot be used for prioritization, but again this is not yet the scope of the proposed solution. -- Michal Hocko SUSE Labs

Re: [v8 0/4] cgroup-aware OOM killer

2017-10-02 Thread Michal Hocko
On Mon 02-10-17 13:24:25, Shakeel Butt wrote: > On Mon, Oct 2, 2017 at 12:56 PM, Michal Hocko wrote: > > On Mon 02-10-17 12:45:18, Shakeel Butt wrote: > >> > I am sorry to cut the rest of your proposal because it simply goes over > >> > the scope of the proposed s

Re: [PATCH] mm,hugetlb,migration: don't migrate kernelcore hugepages

2017-10-03 Thread Michal Hocko
On Tue 03-10-17 07:42:25, Alexandru Moise wrote: > On Mon, Oct 02, 2017 at 06:15:00PM +0200, Michal Hocko wrote: [...] > > I really fail to see why kernel vs. movable zones play any role here. > > Zones should be mostly an implementation detail which userspace > > should

Re: [v9 2/5] mm: implement mem_cgroup_scan_tasks() for the root memory cgroup

2017-10-03 Thread Michal Hocko
this is just a preparatory work for later changes. > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc: Vladimir Davydov > Cc: Johannes Weiner > Cc: Tetsuo Handa > Cc: David Rientjes > Cc: Andrew Morton > Cc: Tejun Heo > Cc: kernel-t...@fb.com > Cc: cgro

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
t;chosen_memcg == INFLIGHT_VICTIM) > + return oc->chosen_memcg; > + > + /* Always begin with the task with the biggest memory footprint */ > + oc->chosen_points = 0; > + oc->chosen_task = NULL; > + mem_cgroup_scan_tasks(oc->chosen_memcg, oo

Re: [v9 4/5] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
line argument would fit better IMHO. > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc: Vladimir Davydov > Cc: Johannes Weiner > Cc: Tetsuo Handa > Cc: David Rientjes > Cc: Andrew Morton > Cc: Tejun Heo > Cc: kernel-t...@fb.com > Cc: cgro..

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
On Tue 03-10-17 13:37:21, Roman Gushchin wrote: > On Tue, Oct 03, 2017 at 01:48:48PM +0200, Michal Hocko wrote: [...] > > Wrt. to the implicit inheritance you brought up in a separate email > > thread [1]. Let me quote > > : after some additional thinking I don't t

Re: [v9 4/5] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
On Tue 03-10-17 13:49:36, Roman Gushchin wrote: > On Tue, Oct 03, 2017 at 01:50:36PM +0200, Michal Hocko wrote: > > On Wed 27-09-17 14:09:35, Roman Gushchin wrote: > > > Add a "groupoom" cgroup v2 mount option to enable the cgroup-aware > > > OOM killer. If no

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
On Tue 03-10-17 15:08:41, Roman Gushchin wrote: > On Tue, Oct 03, 2017 at 03:36:23PM +0200, Michal Hocko wrote: [...] > > I guess we want to inherit the value on the memcg creation but I agree > > that enforcing parent setting is weird. I will think about it some more > > b

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-03 Thread Michal Hocko
On Tue 03-10-17 15:38:08, Roman Gushchin wrote: > On Tue, Oct 03, 2017 at 04:22:46PM +0200, Michal Hocko wrote: > > On Tue 03-10-17 15:08:41, Roman Gushchin wrote: > > > On Tue, Oct 03, 2017 at 03:36:23PM +0200, Michal Hocko wrote: > > [...] > > > > I guess

Re: [v9 3/5] mm, oom: cgroup-aware OOM killer

2017-10-04 Thread Michal Hocko
On Tue 03-10-17 07:35:59, Tejun Heo wrote: > Hello, Michal. > > On Tue, Oct 03, 2017 at 04:22:46PM +0200, Michal Hocko wrote: > > On Tue 03-10-17 15:08:41, Roman Gushchin wrote: > > > On Tue, Oct 03, 2017 at 03:36:23PM +0200, Michal Hocko wrote: > > [...] > >

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
need to call > > oom_evaluate_memcg() for offlined memcgs. > > Sounds like a good optimization, which can be done on top of the current > patchset. You could achive this by checking whether a memcg has tasks rather than explicitly checking for children memcgs as I've suggested

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
the next patch would probably make sense. Although, us reviewers have > been made aware of this now, so I don't feel strongly about it. Won't > make much of a difference once the patches are merged. I think it would be better to move it because it will be less confusing that way. Especially for those who are going to read git history in order to understand why this is needed. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
g tasks. Thanks for separating the group_oom part. This is getting in the mergeable state. I will ack it once the suggested fixes are folded in. There is some clean up potential on top (I find the oc->chosen_memcg quite ugly and will post a patch on top of yours) but that can be done later. >

Re: [v10 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
n_tasks(oc->chosen_memcg, oom_evaluate_task, oc); > + > + if (oc->chosen_task == NULL || > + oc->chosen_task == INFLIGHT_VICTIM) > + goto out; How can this happen? There shouldn't be any INFLIGHT_VICTIM in our memcg because

Re: [v10 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
On Thu 05-10-17 13:32:14, Roman Gushchin wrote: > On Thu, Oct 05, 2017 at 02:06:49PM +0200, Michal Hocko wrote: > > On Wed 04-10-17 16:46:36, Roman Gushchin wrote: > > > The cgroup-aware OOM killer treats leaf memory cgroups as memory > > > consumption entities and perfo

Re: [v10 5/6] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
ave a large base of legacy users yet. I agree on the interface part but I disagree with making it default just because v2 is not largerly adopted yet. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v10 5/6] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
On Thu 05-10-17 14:41:13, Roman Gushchin wrote: > On Thu, Oct 05, 2017 at 03:14:19PM +0200, Michal Hocko wrote: > > On Wed 04-10-17 16:04:53, Johannes Weiner wrote: > > [...] > > > That will silently ignore what the user writes to the memory.oom_group > > > control

Re: [v11 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
group. > > The root cgroup is treated as a leaf memory cgroup, so it's score > is compared with leaf memory cgroups. > Due to memcg statistics implementation a special algorithm > is used for estimating it's oom_score: we define it as maximum > oom_score of the belong

Re: [v11 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
configuration might lead to data corruptions or other > misbehavior. > > The default value is 0. I still believe that oc->chosen_task == INFLIGHT_VICTIM check in oom_kill_memcg_victim should go away. > > Signed-off-by: Roman Gushchin > Cc: Michal Hocko > Cc: Vladim

Re: [v11 4/6] mm, oom: introduce memory.oom_group

2017-10-05 Thread Michal Hocko
score += score; - - if (group_score > oc->chosen_points) { - oc->chosen_points = group_score; - oc->chosen_memcg = group; + if (score > oc->chosen_points) { + oc->chosen_points = score; + oc->chosen_memcg = iter; } } -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v10 5/6] mm, oom: add cgroup v2 mount option for cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
On Thu 05-10-17 10:54:01, Johannes Weiner wrote: > On Thu, Oct 05, 2017 at 03:14:19PM +0200, Michal Hocko wrote: > > On Wed 04-10-17 16:04:53, Johannes Weiner wrote: > > [...] > > > That will silently ignore what the user writes to the memory.oom_group > > >

Re: [v10 3/6] mm, oom: cgroup-aware OOM killer

2017-10-05 Thread Michal Hocko
knob on top of that if you need. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v11 4/6] mm, oom: introduce memory.oom_group

2017-10-06 Thread Michal Hocko
On Fri 06-10-17 13:04:35, Roman Gushchin wrote: > On Thu, Oct 05, 2017 at 04:31:04PM +0200, Michal Hocko wrote: > > Btw. here is how I would do the recursive oom badness. The diff is not > > the nicest one because there is some code moving but the resulting code > > is small

Re: [v11 3/6] mm, oom: cgroup-aware OOM killer

2017-10-10 Thread Michal Hocko
+ if (mem_cgroup_select_oom_victim(oc) && oom_kill_memcg_victim(oc)) { > > + delay = true; > > + goto out; > > + } > > + > > select_bad_process(oc); > > This is racy because mem_cgroup_select_oom_victim() found an eligible > oc-&g

Re: [v11 3/6] mm, oom: cgroup-aware OOM killer

2017-10-11 Thread Michal Hocko
stated several times already that future improvements are possible and cover what you have described already. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [v11 3/6] mm, oom: cgroup-aware OOM killer

2017-10-11 Thread Michal Hocko
On Wed 11-10-17 13:27:44, David Rientjes wrote: > On Wed, 11 Oct 2017, Michal Hocko wrote: > > > > For these reasons: unfair comparison of root mem cgroup usage to bias > > > against that mem cgroup from oom kill in system oom conditions, the > > > ability of

Re: [PATCH 1/2] mm, thp: introduce dedicated transparent huge page allocation interfaces

2017-10-17 Thread Michal Hocko
alloc_transhuge_page_node(node, > + (GFP_TRANSHUGE_LIGHT | __GFP_THISNODE)); > if (!new_page) > goto out_fail; > - prep_transhuge_page(new_page); > > isolated = numamigrate_isolate_page(pgdat, page); > if (!isolated) { > dif

Re: [PATCH 1/2] mm, thp: introduce dedicated transparent huge page allocation interfaces

2017-10-17 Thread Michal Hocko
On Tue 17-10-17 12:20:52, Michal Hocko wrote: > [CC Kirill] now for real > On Mon 16-10-17 17:19:16, changbin...@intel.com wrote: > > From: Changbin Du > > > > This patch introduced 4 new interfaces to allocate a prepared > > transparent huge page. &g

Re: [PATCH 2/2] mm: rename page dtor functions to {compound,huge,transhuge}_page__dtor

2017-10-17 Thread Michal Hocko
t; index 8119270..91d9045 100644 > --- a/mm/userfaultfd.c > +++ b/mm/userfaultfd.c > @@ -323,7 +323,7 @@ static __always_inline ssize_t > __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, >* map of a private mapping, the map was modified to indicate >

Re: [PATCH 2/2] mm: rename page dtor functions to {compound,huge,transhuge}_page__dtor

2017-10-17 Thread Michal Hocko
On Tue 17-10-17 14:22:14, Kirill A. Shutemov wrote: > On Tue, Oct 17, 2017 at 12:22:03PM +0200, Michal Hocko wrote: > > On Mon 16-10-17 17:19:17, changbin...@intel.com wrote: > > > From: Changbin Du > > > > > > The current name free_{huge,transhuge}_pa

Re: [PATCH 1/2] mm, thp: introduce dedicated transparent huge page allocation interfaces

2017-10-19 Thread Michal Hocko
On Wed 18-10-17 19:00:26, Du, Changbin wrote: > Hi Hocko, > > On Tue, Oct 17, 2017 at 12:20:52PM +0200, Michal Hocko wrote: > > [CC Kirill] > > > > On Mon 16-10-17 17:19:16, changbin...@intel.com wrote: > > > From: Changbin Du > > > > > >

Re: [RESEND v12 3/6] mm, oom: cgroup-aware OOM killer

2017-10-19 Thread Michal Hocko
implementation a special approximation > is used for estimating oom_score of root memory cgroup: we sum > oom_score of the belonging processes (or, to be more precise, > tasks owning their mm structures). > > Signed-off-by: Roman Gushchin > Acked-by: Michal Hocko Just to ma

Re: [RESEND v12 0/6] cgroup-aware OOM killer

2017-10-19 Thread Michal Hocko
"only v2 behavior" > Tejun also wasn't convinced > of the risk for regression, and too would prefer cgroup-awareness to > be the default in cgroup2. I would ask for patch 5/6 to be dropped. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "u

Re: [RESEND v12 0/6] cgroup-aware OOM killer

2017-10-23 Thread Michal Hocko
ide itself from the global memory killer by spawning new processes. > (3) the inability of userspace to effectively control oom victim > selection. this is not requested by the current usecase and it has been pointed out that this will be possible to implement on top of the found

Re: [RESEND v12 0/6] cgroup-aware OOM killer

2017-10-31 Thread Michal Hocko
improvements can be implemented without changing user visible behavior as and add-on. If you disagree then you better come with a solid proof that all of us wrong and reasonable semantic cannot be achieved that way. -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "

Re: [RESEND v12 0/6] cgroup-aware OOM killer

2017-10-31 Thread Michal Hocko
emorykiller, > where this is the life-line when the user-space for some reason fails. > > So I guess quite a few will have this problem. Could you be more specific please? We are _not_ removing possibility of the user space influenced oom victim selection. You can still use the _current_ oom selection heuristic. The patch adds a new selection method which is opt-in so only those who want to opt-in will not be allowed to have any influence on the victim selection. And as it has been pointed out this can be implemented later so it is not like "this won't be possible anymore in future" -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [RESEND v12 3/6] mm, oom: cgroup-aware OOM killer

2017-10-31 Thread Michal Hocko
charge migration in v2. To be honest I wasn't completely happy about removing this functionality altogether in v2 but there was a strong pushback back then that relying on the charge migration doesn't have any sound usecase. Anyway, I agree that documentation should be explicit about that. -- M

Re: [RESEND v12 3/6] mm, oom: cgroup-aware OOM killer

2017-10-31 Thread Michal Hocko
On Tue 31-10-17 16:29:23, Michal Hocko wrote: > On Tue 31-10-17 08:04:19, Shakeel Butt wrote: > > > + > > > +static void select_victim_memcg(struct mem_cgroup *root, struct > > > oom_control *oc) > > > +{ > > > + struct mem_cgroup *ite

Re: [RESEND v12 3/6] mm, oom: cgroup-aware OOM killer

2017-10-31 Thread Michal Hocko
On Tue 31-10-17 20:06:44, Michal Hocko wrote: > On Tue 31-10-17 16:29:23, Michal Hocko wrote: > > On Tue 31-10-17 08:04:19, Shakeel Butt wrote: > > > > + > > > > +static void select_victim_memcg(struct mem_cgroup *root, struct > > > > oom_control *oc)

Re: [RESEND v12 0/6] cgroup-aware OOM killer

2017-11-01 Thread Michal Hocko
On Tue 31-10-17 15:21:23, David Rientjes wrote: > On Tue, 31 Oct 2017, Michal Hocko wrote: > > > > I'm not ignoring them, I have stated that we need the ability to protect > > > important cgroups on the system without oom disabling all attached > > > p

Re: [PATCH v5 0/3] mm, proc: Implement /proc//totmaps

2016-09-19 Thread Michal Hocko
On Mon 19-09-16 11:16:31, Robert Foss wrote: > On 2016-09-14 05:12 AM, Michal Hocko wrote: > > On Tue 13-09-16 13:27:39, Sonny Rao wrote: [...] > > > Given that smaps > > > doesn't provide this in a straightforward way, what do you think is > > >

Re: utime accounting regression since 4.6 (was: Re: [PACTH v2 0/3] Implement /proc//totmaps)

2016-09-30 Thread Michal Hocko
8-23 at 16:33 +0200, Michal Hocko wrote: [...] > > OK, so it seems I found it. I was quite lucky because > > account_user_time > > is not all that popular function and there were basically no changes > > besides Riks ff9a9b4c4334 ("sched, time: Switch > > VIRT_CPU_AC

Re: [RFC PATCH v3] sparc64: Add support for Application Data Integrity (ADI)

2017-01-06 Thread Michal Hocko
larity sounds way too tricky... -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-22 Thread Michal Hocko
ing it difficult to use. HugeTLBfs > may support multiple huge page sizes, and in such a special environment > there is a desire to use HugeTLBfs. Well, then I would argue that you shouldn't use 64kB pages for your setup or allow THP for smaller sizes. Really hugetlb pages are by no

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-22 Thread Michal Hocko
detest a possibility to put hugetlb pages into the memcg mix. They just do not belong there. Try to look at previous discussions why it has been decided to have a separate hugetlb pages at all. I am also quite confused why you keep distinguishing surplus hugetlb pages from regular prea

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 13:39:59, TSUKADA Koutaro wrote: > On 2018/05/23 3:54, Michal Hocko wrote: [...] > > I am also quite confused why you keep distinguishing surplus hugetlb > > pages from regular preallocated ones. Being a surplus page is an > > implementation detail that w

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-24 Thread Michal Hocko
commitment will allow users to exhaust > the entire memory of the system. Of course, this can be prevented by the > hugetlb cgroup, but even if we set the limit for memcg and hugetlb cgroup > respectively, as I asked in the first mail(set limit to 10GB), the > control will n

Re: [PATCH v2 0/7] mm: pages for hugetlb's overcommit may be able to charge to memcg

2018-05-24 Thread Michal Hocko
On Thu 24-05-18 21:58:49, TSUKADA Koutaro wrote: > On 2018/05/24 17:20, Michal Hocko wrote: > > On Thu 24-05-18 13:39:59, TSUKADA Koutaro wrote: > >> On 2018/05/23 3:54, Michal Hocko wrote: > > [...] > >>> I am also quite confused why you keep distinguish

Re: [PATCH 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide

2018-12-05 Thread Michal Hocko
min guide. It is a pure implementation detail nobody should be relying on. -- Michal Hocko SUSE Labs

Re: [PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks

2018-12-05 Thread Michal Hocko
hotplug (e.g. access to global/zone > -variables). > +variables). Currently, we take advantage of this to serialise sparsemem's > +mem_section handling in sparse_add_one_section() and > +sparse_remove_one_section(). > > In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > mode allows for a quite efficient get_online_mems/put_online_mems > -- > 2.15.1 > -- Michal Hocko SUSE Labs

Re: [PATCH v2] Document /proc/pid PID reuse behavior

2018-11-06 Thread Michal Hocko
to be named the same numeric PID. This sounds quite obvious otherwise anybody could simply DoS the system by consuming all available pids. But I do not see any strong reason against having that stated explicitly. > Signed-off-by: Daniel Colascione Acked-by: Michal Hocko > --- > Docu

Re: [PATCH v2] Document /proc/pid PID reuse behavior

2018-11-07 Thread Michal Hocko
On Wed 07-11-18 15:48:20, Daniel Colascione wrote: > On Tue, Nov 6, 2018 at 1:05 PM, Michal Hocko wrote: > > On Mon 05-11-18 13:22:05, Daniel Colascione wrote: > >> State explicitly that holding a /proc/pid file descriptor open does > >> not reserve the PID. Also no

<    1   2   3   >