Re: [PATCH] swap: redirty page if page write fails on swap file

2013-05-01 Thread Simon Jeons
Ping, ;-) On 04/18/2013 08:13 AM, Simon Jeons wrote: Hi Jerome, On 04/17/2013 08:11 PM, Jerome Marchand wrote: Since commit 62c230b, swap_writepage() calls direct_IO on swap files. However, in that case page isn't redirtied if I/O fails, and is therefore handled afterwards as if it has

Re: [PATCH 1/2] Make the batch size of the percpu_counter configurable

2013-04-30 Thread Simon Jeons
Hi Tim, On 04/30/2013 01:12 AM, Tim Chen wrote: Currently, there is a single, global, variable (percpu_counter_batch) that controls the batch sizes for every 'struct percpu_counter' on the system. However, there are some applications, e.g. memory accounting where it is more appropriate to scale

Re: [PATCH] mm: cond_resched in tlb_flush_mmu to fix soft lockups on !CONFIG_PREEMPT

2013-04-27 Thread Simon Jeons
Hi Michal, On 12/19/2012 12:11 AM, Michal Hocko wrote: Since e303297 (mm: extended batches for generic mmu_gather) we are batching pages to be freed until either tlb_next_batch cannot allocate a new batch or we are done. Is there material introduce mmu_gather? This works just fine most of th

Re: [PATCH 0/10] Reduce system disruption due to kswapd V2

2013-04-22 Thread Simon Jeons
Hi Zlatko, On 04/22/2013 02:54 PM, Zlatko Calusic wrote: On 22.04.2013 08:43, Simon Jeons wrote: Hi Zlatko, On 04/22/2013 02:37 PM, Zlatko Calusic wrote: On 12.04.2013 22:07, Zlatko Calusic wrote: On 12.04.2013 21:40, Mel Gorman wrote: On Thu, Apr 11, 2013 at 10:55:13PM +0200, Zlatko Calusic

Re: [PATCH 0/10] Reduce system disruption due to kswapd V2

2013-04-21 Thread Simon Jeons
Hi Zlatko, On 04/22/2013 02:37 PM, Zlatko Calusic wrote: On 12.04.2013 22:07, Zlatko Calusic wrote: On 12.04.2013 21:40, Mel Gorman wrote: On Thu, Apr 11, 2013 at 10:55:13PM +0200, Zlatko Calusic wrote: On 09.04.2013 13:06, Mel Gorman wrote: - The only slightly negative thing I observed is t

Re: [RFC PATCH v2 00/15][Sorted-buddy] mm: Memory Power Management

2013-04-18 Thread Simon Jeons
Hi Srivatsa, On 04/10/2013 05:45 AM, Srivatsa S. Bhat wrote: [I know, this cover letter is a little too long, but I wanted to clearly explain the overall goals and the high-level design of this patchset in detail. I hope this helps more than it annoys, and makes it easier for reviewers to relate

Re: [RFC Patch 0/2] mm: Add parameters to make kernel behavior at memory error on dirty cache selectable

2013-04-17 Thread Simon Jeons
Hi Naoya, On 04/17/2013 10:55 PM, Naoya Horiguchi wrote: > On Wed, Apr 17, 2013 at 03:14:36PM +0800, Simon Jeons wrote: >> Hi Naoya, >> On 04/11/2013 03:11 PM, Naoya Horiguchi wrote: >>> Hi Tanino-san, >>> >>> On Thu, Apr 11, 2013 at 12:26:19PM +0900, Mit

Re: [PATCH] swap: redirty page if page write fails on swap file

2013-04-17 Thread Simon Jeons
Hi Jerome, On 04/17/2013 08:11 PM, Jerome Marchand wrote: Since commit 62c230b, swap_writepage() calls direct_IO on swap files. However, in that case page isn't redirtied if I/O fails, and is therefore handled afterwards as if it has been successfully written to the swap file, leading to memory c

Re: [PATCH] mm: mmu_notifier: re-fix freed page still mapped in secondary MMU

2013-04-17 Thread Simon Jeons
Hi Robin, On 04/16/2013 05:31 PM, Robin Holt wrote: On Tue, Apr 16, 2013 at 02:39:49PM +0800, Xiao Guangrong wrote: The commit 751efd8610d3 (mmu_notifier_unregister NULL Pointer deref and multiple ->release()) breaks the fix: 3ad3d901bbcfb15a5e4690e55350db0899095a68 (mm: mmu_notifier:

Re: [RFC Patch 0/2] mm: Add parameters to make kernel behavior at memory error on dirty cache selectable

2013-04-17 Thread Simon Jeons
Hi Naoya, On 04/11/2013 03:11 PM, Naoya Horiguchi wrote: > Hi Tanino-san, > > On Thu, Apr 11, 2013 at 12:26:19PM +0900, Mitsuhiro Tanino wrote: > ... >> Solution >> - >> The patch proposes a new sysctl interface, vm.memory_failure_dirty_panic, >> in order to prevent data corruption comes fr

Re: [RFC Patch 0/2] mm: Add parameters to make kernel behavior at memory error on dirty cache selectable

2013-04-16 Thread Simon Jeons
Hi Naoya, On 04/11/2013 11:23 PM, Naoya Horiguchi wrote: > On Thu, Apr 11, 2013 at 03:49:16PM +0200, Andi Kleen wrote: >>> As a result, if the dirty cache includes user data, the data is lost, >>> and data corruption occurs if an application uses old data. >> The application cannot use old data, th

Re: [RFC Patch 0/2] mm: Add parameters to make kernel behavior at memory error on dirty cache selectable

2013-04-16 Thread Simon Jeons
Hi Mitsuhiro, On 04/12/2013 09:43 PM, Mitsuhiro Tanino wrote: (2013/04/11 22:00), Ric Mason wrote: Hi Mitsuhiro, On 04/11/2013 08:51 PM, Mitsuhiro Tanino wrote: (2013/04/11 12:53), Simon Jeons wrote: One question against mce instead of the patchset. ;-) When check memory is bad? Before

Re: [RFC Patch 0/2] mm: Add parameters to make kernel behavior at memory error on dirty cache selectable

2013-04-16 Thread Simon Jeons
On 04/11/2013 09:49 PM, Andi Kleen wrote: As a result, if the dirty cache includes user data, the data is lost, and data corruption occurs if an application uses old data. Hi Andi, Could you give me the link of your mce testcase? The application cannot use old data, the kernel code kills it

Re: [BUG][s390x] mm: system crashed

2013-04-16 Thread Simon Jeons
Hi Heiko, On 04/16/2013 03:50 PM, Heiko Carstens wrote: On Mon, Apr 15, 2013 at 02:16:55PM +0800, Zhouping Liu wrote: On 04/15/2013 01:56 PM, Heiko Carstens wrote: On Sun, Apr 14, 2013 at 11:28:40PM -0400, Zhouping Liu wrote: � 16109.346170¨ Call Trace: � 16109.346179¨ (� <00100920>¨ s

Re: [RFC Patch 0/2] mm: Add parameters to make kernel behavior at memory error on dirty cache selectable

2013-04-10 Thread Simon Jeons
Hi Mitsuhiro, On 04/11/2013 11:26 AM, Mitsuhiro Tanino wrote: Hi All, Please find a patch set that introduces these new sysctl interfaces, to handle a case when an memory error is detected on dirty page cache. - vm.memory_failure_dirty_panic - vm.memory_failure_print_ratelimit - vm.memory_failur

Re: [PATCH 2/3] mm, slub: count freed pages via rcu as this task's reclaimed_slab

2013-04-10 Thread Simon Jeons
Hi Christoph, On 04/10/2013 09:54 PM, Christoph Lameter wrote: On Wed, 10 Apr 2013, Simon Jeons wrote: It seems that you misunderstand my question. I don't doubt slab/slub can use high order pages. However, what I focus on is why slab/slub can use compound page, PageCompound() just on b

Re: [PATCH] x86: add phys addr validity check for /dev/mem mmap

2013-04-10 Thread Simon Jeons
Hi H.Peter, On 04/11/2013 10:48 AM, H. Peter Anvin wrote: On 04/10/2013 07:40 PM, Simon Jeons wrote: Hi H.Peter, On 04/04/2013 09:13 AM, H. Peter Anvin wrote: On 04/03/2013 06:11 PM, Simon Jeons wrote: Why we consider boot_cpu_data.x86_phys_bits instead of e820 map here? Because

Re: [PATCH] x86: add phys addr validity check for /dev/mem mmap

2013-04-10 Thread Simon Jeons
Hi H.Peter, On 04/04/2013 09:13 AM, H. Peter Anvin wrote: On 04/03/2013 06:11 PM, Simon Jeons wrote: Why we consider boot_cpu_data.x86_phys_bits instead of e820 map here? Because x86_phys_bits is what controls how much address space the processor has. e820 tells us how much *RAM* the machine

Re: [PATCH 2/3] mm, slub: count freed pages via rcu as this task's reclaimed_slab

2013-04-09 Thread Simon Jeons
Hi Christoph, On 04/09/2013 10:32 PM, Christoph Lameter wrote: On Tue, 9 Apr 2013, Simon Jeons wrote: + int pages = 1 << compound_order(page); One question irrelevant this patch. Why slab cache can use compound page(hugetlbfs pages/thp pages)? They are just used by app to optimi

Re: [PATCH v8 3/3] mm: reinititalise user and admin reserves if memory is added or removed

2013-04-09 Thread Simon Jeons
On 04/10/2013 08:11 AM, Andrew Shewmaker wrote: On Tue, Apr 9, 2013 at 6:05 PM, Simon Jeons wrote: Hi Andrew, On 04/10/2013 07:56 AM, Andrew Shewmaker wrote: On Tue, Apr 9, 2013 at 4:19 PM, Andrew Morton wrote: On Mon, 8 Apr 2013 17:00:40 -0400 Andrew Shewmaker wrote: Should I add the

Re: [PATCH v8 3/3] mm: reinititalise user and admin reserves if memory is added or removed

2013-04-09 Thread Simon Jeons
Hi Andrew, On 04/10/2013 07:56 AM, Andrew Shewmaker wrote: On Tue, Apr 9, 2013 at 4:19 PM, Andrew Morton wrote: On Mon, 8 Apr 2013 17:00:40 -0400 Andrew Shewmaker wrote: Should I add the memory notifier code to mm/nommu.c too? I'm guessing that if a system doesn't have an mmu that it also wo

Re: [PATCH] mm: page_alloc: Avoid marking zones full prematurely after zone_reclaim()

2013-04-09 Thread Simon Jeons
Hi Michal, On 04/09/2013 06:14 PM, Michal Hocko wrote: On Tue 09-04-13 18:05:30, Simon Jeons wrote: [...] I try this in v3.9-rc5: dd if=/dev/sda of=/dev/null bs=1MB 14813+0 records in 14812+0 records out 1481200 bytes (15 GB) copied, 105.988 s, 140 MB/s free -m -s 1

Re: [PATCH] mm: page_alloc: Avoid marking zones full prematurely after zone_reclaim()

2013-04-09 Thread Simon Jeons
Hi Michal, On 04/05/2013 02:31 PM, Simon Jeons wrote: Hi Michal, On 03/21/2013 04:19 PM, Michal Hocko wrote: On Thu 21-03-13 10:33:07, Simon Jeons wrote: Hi Mel, On 03/21/2013 02:19 AM, Mel Gorman wrote: The following problem was reported against a distribution kernel when zone_reclaim was

Re: [PATCH 2/3] mm, slub: count freed pages via rcu as this task's reclaimed_slab

2013-04-09 Thread Simon Jeons
Hi Joonsoo, On 04/09/2013 09:21 AM, Joonsoo Kim wrote: Currently, freed pages via rcu is not counted for reclaimed_slab, because it is freed in rcu context, not current task context. But, this free is initiated by this task, so counting this into this task's reclaimed_slab is meaningful to decide

Re: [PATCH 08/10] mm: vmscan: Have kswapd shrink slab only once per priority

2013-04-09 Thread Simon Jeons
Hi Joonsoo, On 04/09/2013 02:53 PM, Joonsoo Kim wrote: Hello, Mel. Sorry for too late question. On Sun, Mar 17, 2013 at 01:04:14PM +, Mel Gorman wrote: If kswaps fails to make progress but continues to shrink slab then it'll either discard all of slab or consume CPU uselessly scanning shrin

Re: [PATCH 1/3] mm/page_alloc: factor out setting of pcp->high and pcp->batch.

2013-04-08 Thread Simon Jeons
Hi Cody, On 04/09/2013 01:39 AM, Cody P Schafer wrote: On 04/06/2013 06:37 PM, Simon Jeons wrote: Hi Cody, On 04/06/2013 04:33 AM, Cody P Schafer wrote: Creates pageset_set_batch() for use in setup_pageset(). pageset_set_batch() imitates the functionality of setup_pagelist_highmark(), but uses

Re: [RFC] mm: remove swapcache page early

2013-04-07 Thread Simon Jeons
On 04/08/2013 09:48 AM, Minchan Kim wrote: Hello Simon, On Sun, Apr 07, 2013 at 03:26:12PM +0800, Simon Jeons wrote: Ping Minchan. On 04/02/2013 09:40 PM, Simon Jeons wrote: Hi Hugh, On 03/28/2013 05:41 AM, Hugh Dickins wrote: On Wed, 27 Mar 2013, Minchan Kim wrote: Swap subsystem does

Re: [RFC] mm: remove swapcache page early

2013-04-07 Thread Simon Jeons
Ping Minchan. On 04/02/2013 09:40 PM, Simon Jeons wrote: Hi Hugh, On 03/28/2013 05:41 AM, Hugh Dickins wrote: On Wed, 27 Mar 2013, Minchan Kim wrote: Swap subsystem does lazy swap slot free with expecting the page would be swapped out again so we can't avoid unnecessary

Re: [PATCH] mm: page_alloc: Avoid marking zones full prematurely after zone_reclaim()

2013-04-06 Thread Simon Jeons
Ping! On 04/05/2013 02:31 PM, Simon Jeons wrote: Hi Michal, On 03/21/2013 04:19 PM, Michal Hocko wrote: On Thu 21-03-13 10:33:07, Simon Jeons wrote: Hi Mel, On 03/21/2013 02:19 AM, Mel Gorman wrote: The following problem was reported against a distribution kernel when zone_reclaim was enabled

Re: [PATCH 3/3] mm: when handling percpu_pagelist_fraction, use on_each_cpu() to set percpu pageset fields.

2013-04-06 Thread Simon Jeons
Hi Cody, On 04/06/2013 04:33 AM, Cody P Schafer wrote: In free_hot_cold_page(), we rely on pcp->batch remaining stable. Updating it without being on the cpu owning the percpu pageset potentially destroys this stability. If cpu is off, can its pcp pageset be used in free_hot_code_page()? Chan

Re: [PATCH 1/3] mm/page_alloc: factor out setting of pcp->high and pcp->batch.

2013-04-06 Thread Simon Jeons
Hi Cody, On 04/06/2013 04:33 AM, Cody P Schafer wrote: Creates pageset_set_batch() for use in setup_pageset(). pageset_set_batch() imitates the functionality of setup_pagelist_highmark(), but uses the boot time (percpu_pagelist_fraction == 0) calculations for determining ->high Why need adjust

Re: [PATCH v4, part3 00/15] accurately calculate memory statisitic information

2013-04-06 Thread Simon Jeons
Hi Jiang, On 04/06/2013 09:54 PM, Jiang Liu wrote: The original goal of this patchset is to fix the bug reported by https://bugzilla.kernel.org/show_bug.cgi?id=53501 Now it has also been expanded to reduce common code used by memory initializion. This is the third part, previous two patch sets c

Re: [PATCH 0/3] mm: fixup changers of per cpu pageset's ->high and ->batch

2013-04-06 Thread Simon Jeons
Hi Cody, On 04/06/2013 04:33 AM, Cody P Schafer wrote: In one case while modifying the ->high and ->batch fields of per cpu pagesets we're unneededly using stop_machine() (patches 1 & 2), and in another we don't have any syncronization at all (patch 3). Do you mean stop_machine() is used for s

Re: [PATCH v7 2/2] mm: replace hardcoded 3% with admin_reserve_pages knob

2013-04-06 Thread Simon Jeons
Hi Andrew, On 04/05/2013 11:02 PM, Andrew Shewmaker wrote: On Wed, Apr 3, 2013 at 9:50 PM, Simon Jeons wrote: FAQ ... * How do you calculate a minimum useful reserve? A user or the admin needs enough memory to login and perform recovery operations, which includes, at a minimum

Re: [RFC][PATCH 0/9] extend hugepage migration

2013-04-06 Thread Simon Jeons
Hi Michal, On 04/05/2013 05:30 PM, Michal Hocko wrote: On Fri 05-04-13 17:00:58, Simon Jeons wrote: Hi Michal, On 04/05/2013 04:08 PM, Michal Hocko wrote: On Fri 05-04-13 09:14:58, Simon Jeons wrote: Hi Michal, On 03/22/2013 04:15 PM, Michal Hocko wrote: [getting off-list] On Fri 22-03-13

Re: [RFC][PATCH 0/9] extend hugepage migration

2013-04-05 Thread Simon Jeons
Hi Michal, On 04/05/2013 04:08 PM, Michal Hocko wrote: On Fri 05-04-13 09:14:58, Simon Jeons wrote: Hi Michal, On 03/22/2013 04:15 PM, Michal Hocko wrote: [getting off-list] On Fri 22-03-13 07:46:32, Simon Jeons wrote: Hi Michal, On 03/21/2013 08:56 PM, Michal Hocko wrote: On Thu 21-03-13

Re: [PATCH 0/6] mm/hugetlb: gigantic hugetlb page pools shrink supporting

2013-04-05 Thread Simon Jeons
Hi Michal, On 04/05/2013 04:12 PM, Michal Hocko wrote: On Fri 05-04-13 07:41:23, Wanpeng Li wrote: On Thu, Apr 04, 2013 at 06:17:46PM +0200, Michal Hocko wrote: On Thu 04-04-13 17:09:08, Wanpeng Li wrote: order >= MAX_ORDER pages are only allocated at boot stage using the bootmem allocator wit

Re: [PATCHv2, RFC 20/30] ramfs: enable transparent huge page cache

2013-04-04 Thread Simon Jeons
Hi Minchan, On 04/03/2013 09:11 AM, Minchan Kim wrote: On Tue, Apr 02, 2013 at 03:15:23PM -0700, Hugh Dickins wrote: On Tue, 2 Apr 2013, Kirill A. Shutemov wrote: Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" ramfs is the most simple fs from page cache point of view. Let's start trans

Re: [PATCH] mm: page_alloc: Avoid marking zones full prematurely after zone_reclaim()

2013-04-04 Thread Simon Jeons
Hi Michal, On 03/21/2013 04:19 PM, Michal Hocko wrote: On Thu 21-03-13 10:33:07, Simon Jeons wrote: Hi Mel, On 03/21/2013 02:19 AM, Mel Gorman wrote: The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline

Re: [RFC][PATCH 0/9] extend hugepage migration

2013-04-04 Thread Simon Jeons
Hi Michal, On 03/22/2013 04:15 PM, Michal Hocko wrote: [getting off-list] On Fri 22-03-13 07:46:32, Simon Jeons wrote: Hi Michal, On 03/21/2013 08:56 PM, Michal Hocko wrote: On Thu 21-03-13 07:49:48, Simon Jeons wrote: [...] When I hacking arch/x86/mm/hugetlbpage.c like this, diff --git a

Re: [PATCH, RFC 00/16] Transparent huge page cache

2013-04-04 Thread Simon Jeons
Hi Hugh, On 01/31/2013 10:12 AM, Hugh Dickins wrote: On Tue, 29 Jan 2013, Kirill A. Shutemov wrote: Hugh Dickins wrote: On Mon, 28 Jan 2013, Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" Here's first steps towards huge pages in page cache. The intend of the work is get code ready to

Re: [PATCH, RFC 00/16] Transparent huge page cache

2013-04-04 Thread Simon Jeons
Hi Hugh, On 01/31/2013 10:12 AM, Hugh Dickins wrote: On Tue, 29 Jan 2013, Kirill A. Shutemov wrote: Hugh Dickins wrote: On Mon, 28 Jan 2013, Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" Here's first steps towards huge pages in page cache. The intend of the work is get code ready to

Re: [PATCH] x86: add phys addr validity check for /dev/mem mmap

2013-04-03 Thread Simon Jeons
Hi H.Peter, On 04/04/2013 09:32 AM, H. Peter Anvin wrote: On 04/03/2013 06:17 PM, Simon Jeons wrote: e820 also contain mmio, correct? No. How to check which address is used by mmio? /proc/iomem, correct? So cpu should not access address beyond e820 map(RAM+MMIO). No. -hpa

Re: [PATCH 1/9] migrate: add migrate_entry_wait_huge()

2013-04-03 Thread Simon Jeons
Ping! On 03/21/2013 07:36 AM, Simon Jeons wrote: > Hi Naoya, > On 03/21/2013 05:53 AM, Naoya Horiguchi wrote: >> On Wed, Mar 20, 2013 at 07:57:32AM +0800, Simon Jeons wrote: >>> Hi Naoya, >>> On 02/22/2013 03:41 AM, Naoya Horiguchi wrote: >>>> When we

Re: [PATCH v7 2/2] mm: replace hardcoded 3% with admin_reserve_pages knob

2013-04-03 Thread Simon Jeons
rt programmer will recognize this and will adapt the setting using getpagesize(2), but if we define these things in "bytes" rather than "pages" then dumb programmers can use it too. -Andrew Morton v4: * Rebased onto v3.8-mmotm-2013-03-01-15-50 * No longer assumes

Re: [PATCH] THP: Use explicit memory barrier

2013-04-03 Thread Simon Jeons
Hi Minchan, On 04/01/2013 07:45 AM, Minchan Kim wrote: __do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write. 1. There are no pte modify, why take page_table_lock here? 2. What's the meani

Re: [PATCH] x86: add phys addr validity check for /dev/mem mmap

2013-04-03 Thread Simon Jeons
On 04/04/2013 10:14 AM, H. Peter Anvin wrote: Because git didn't exist before then? Oh, I see, thanks! :-) Simon Jeons wrote: On 04/04/2013 09:32 AM, H. Peter Anvin wrote: On 04/03/2013 06:17 PM, Simon Jeons wrote: e820 also contain mmio, correct? No. So cpu should not a

Re: [PATCH] x86: add phys addr validity check for /dev/mem mmap

2013-04-03 Thread Simon Jeons
On 04/04/2013 09:32 AM, H. Peter Anvin wrote: On 04/03/2013 06:17 PM, Simon Jeons wrote: e820 also contain mmio, correct? No. So cpu should not access address beyond e820 map(RAM+MMIO). No. -hpa One offline question, why can't check git log before 2005? -- To unsubscribe

Re: [PATCH] x86: add phys addr validity check for /dev/mem mmap

2013-04-03 Thread Simon Jeons
Hi H.Peter, On 04/04/2013 09:13 AM, H. Peter Anvin wrote: On 04/03/2013 06:11 PM, Simon Jeons wrote: Why we consider boot_cpu_data.x86_phys_bits instead of e820 map here? Because x86_phys_bits is what controls how much address space the processor has. e820 tells us how much *RAM* the machine

Re: [PATCH] x86: add phys addr validity check for /dev/mem mmap

2013-04-03 Thread Simon Jeons
Hi H.Peter, On 04/03/2013 02:48 AM, H. Peter Anvin wrote: On 04/02/2013 05:28 AM, Frantisek Hrbata wrote: diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index d8e8eef..39607c6 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -242,6 +242,10 @@ static i

Re: [PATCH] mm, x86: no zeroing of hugetlbfs pages at boot

2013-04-03 Thread Simon Jeons
On 03/07/2013 05:50 AM, Cliff Wickman wrote: From: Cliff Wickman Allocating a large number of 1GB hugetlbfs pages at boot takes a very long time. Large system sites would at times like to allocate a very large amount of memory as 1GB pages. They would put this on the kernel boot line: def

Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?

2013-04-02 Thread Simon Jeons
Hi David, On 04/03/2013 02:09 AM, David Rientjes wrote: On Tue, 2 Apr 2013, Simon Jeons wrote: Both thp and hugetlb pages should be 2MB aligned, correct? To answer this question and your followup reply at the same time: they come from one level higher in the page table so they will naturally

Re: [RFC] mm: remove swapcache page early

2013-04-02 Thread Simon Jeons
Hi Hugh, On 03/28/2013 05:41 AM, Hugh Dickins wrote: On Wed, 27 Mar 2013, Minchan Kim wrote: Swap subsystem does lazy swap slot free with expecting the page would be swapped out again so we can't avoid unnecessary write. so we can avoid unnecessary write. If pag

Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?

2013-04-02 Thread Simon Jeons
On 04/02/2013 06:23 AM, David Rientjes wrote: On Mon, 1 Apr 2013, Zhouping Liu wrote: Hi all, I found THP can't correctly distinguish one anonymous hugepage map. 1. when /sys/kernel/mm/transparent_hugepage/enabled is 'always', the amount of THP always is one less. It's not a problem wit

Re: THP: AnonHugePages in /proc/[pid]/smaps is correct or not?

2013-04-02 Thread Simon Jeons
Hi David, On 04/02/2013 06:23 AM, David Rientjes wrote: On Mon, 1 Apr 2013, Zhouping Liu wrote: Hi all, I found THP can't correctly distinguish one anonymous hugepage map. 1. when /sys/kernel/mm/transparent_hugepage/enabled is 'always', the amount of THP always is one less. It's not a p

Re: [RFC][PATCH 0/9] extend hugepage migration

2013-03-21 Thread Simon Jeons
Hi Michal, On 03/21/2013 08:56 PM, Michal Hocko wrote: On Thu 21-03-13 07:49:48, Simon Jeons wrote: [...] When I hacking arch/x86/mm/hugetlbpage.c like this, diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index ae1aa71..87f34ee 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b

Re: [PATCH] mm: page_alloc: Avoid marking zones full prematurely after zone_reclaim()

2013-03-21 Thread Simon Jeons
Hi Michal, On 03/21/2013 04:19 PM, Michal Hocko wrote: On Thu 21-03-13 10:33:07, Simon Jeons wrote: Hi Mel, On 03/21/2013 02:19 AM, Mel Gorman wrote: The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline

Re: [PATCH, RFC 00/16] Transparent huge page cache

2013-03-21 Thread Simon Jeons
On 01/28/2013 05:24 PM, Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" Here's first steps towards huge pages in page cache. The intend of the work is get code ready to enable transparent huge page cache for the most simple fs -- ramfs. It's not yet near feature-complete. It only provide

Re: [PATCH] mm: page_alloc: Avoid marking zones full prematurely after zone_reclaim()

2013-03-20 Thread Simon Jeons
Hi Mel, On 03/21/2013 02:19 AM, Mel Gorman wrote: The following problem was reported against a distribution kernel when zone_reclaim was enabled but the same problem applies to the mainline kernel. The reproduction case was as follows 1. Run numactl -m +0 dd if=largefile of=/dev/null This al

Re: [PATCH 5/9] migrate: enable migrate_pages() to migrate hugepage

2013-03-20 Thread Simon Jeons
Hi Naoya, On 03/21/2013 05:59 AM, Naoya Horiguchi wrote: > On Wed, Mar 20, 2013 at 08:31:06AM +0800, Simon Jeons wrote: > ... >>>>> diff --git v3.8.orig/mm/mempolicy.c v3.8/mm/mempolicy.c >>>>> index e2df1c1..8627135 100644 >>>>> --- v3.

Re: [PATCH 8/9] memory-hotplug: enable memory hotplug to handle hugepage

2013-03-20 Thread Simon Jeons
Hi Naoya, On 03/21/2013 06:05 AM, Naoya Horiguchi wrote: > On Wed, Mar 20, 2013 at 09:03:20AM +0800, Simon Jeons wrote: >> Hi Naoya, >> On 02/22/2013 03:41 AM, Naoya Horiguchi wrote: >>> Currently we can't offline memory blocks which contain hugepages because >&

Re: [RFC][PATCH 0/9] extend hugepage migration

2013-03-20 Thread Simon Jeons
Hi Naoya, On 03/21/2013 05:35 AM, Naoya Horiguchi wrote: > On Wed, Mar 20, 2013 at 07:43:44AM +0800, Simon Jeons wrote: > ... >>> Easy patch access: >>> g...@github.com:Naoya-Horiguchi/linux.git >>> branch:extend_hugepage_migration >>> >>&g

Re: [PATCH 1/9] migrate: add migrate_entry_wait_huge()

2013-03-20 Thread Simon Jeons
Hi Naoya, On 03/21/2013 05:53 AM, Naoya Horiguchi wrote: > On Wed, Mar 20, 2013 at 07:57:32AM +0800, Simon Jeons wrote: >> Hi Naoya, >> On 02/22/2013 03:41 AM, Naoya Horiguchi wrote: >>> When we have a page fault for the address which is backed by a hugepage >>>

Re: [PATCHv2, RFC 00/30] Transparent huge page cache

2013-03-19 Thread Simon Jeons
Hi Kirill, On 03/15/2013 01:50 AM, Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" Here's the second version of the patchset. The intend of the work is get code ready to enable transparent huge page cache for the most simple fs -- ramfs. We have read()/write()/mmap() functionality now. S

Re: [PATCH 8/9] memory-hotplug: enable memory hotplug to handle hugepage

2013-03-19 Thread Simon Jeons
Hi Naoya, On 02/22/2013 03:41 AM, Naoya Horiguchi wrote: Currently we can't offline memory blocks which contain hugepages because a hugepage is considered as an unmovable page. But now with this patch series, a hugepage has become movable, so by using hugepage migration we can offline such memory

Re: [PATCH 5/9] migrate: enable migrate_pages() to migrate hugepage

2013-03-19 Thread Simon Jeons
Hi Naoya, On 03/19/2013 08:07 AM, Naoya Horiguchi wrote: > On Mon, Mar 18, 2013 at 04:40:57PM +0100, Michal Hocko wrote: >> On Thu 21-02-13 14:41:44, Naoya Horiguchi wrote: >>> This patch extends check_range() to handle vma with VM_HUGETLB set. >>> With this changes, we can migrate hugepage with mi

Re: [PATCH 1/9] migrate: add migrate_entry_wait_huge()

2013-03-19 Thread Simon Jeons
Hi Naoya, On 02/22/2013 03:41 AM, Naoya Horiguchi wrote: When we have a page fault for the address which is backed by a hugepage under migration, the kernel can't wait correctly until the migration finishes. This is because pte_offset_map_lock() can't get a correct It seems that current hugetlb

Re: [RFC][PATCH 0/9] extend hugepage migration

2013-03-19 Thread Simon Jeons
Hi Naoya, On 02/22/2013 03:41 AM, Naoya Horiguchi wrote: Hi, Hugepage migration is now available only for soft offlining (moving data on the half corrupted page to another page to save the data). But it's also useful some other users of page migration, so this patchset tries to extend some of su

Re: [PATCH 03/10] mm: vmscan: Flatten kswapd priority loop

2013-03-19 Thread Simon Jeons
Hi Mel, On 03/19/2013 06:14 PM, Mel Gorman wrote: On Tue, Mar 19, 2013 at 11:08:23AM +0800, Simon Jeons wrote: Hi Mel, On 03/17/2013 09:04 PM, Mel Gorman wrote: kswapd stops raising the scanning priority when at least SWAP_CLUSTER_MAX pages have been reclaimed or the pgdat is considered

Re: [PATCH 01/10] mm: vmscan: Limit the number of pages kswapd reclaims at each priority

2013-03-19 Thread Simon Jeons
Hi Mel, On 03/19/2013 05:55 PM, Mel Gorman wrote: On Tue, Mar 19, 2013 at 07:53:16AM +0800, Simon Jeons wrote: Hi Mel, On 03/17/2013 09:04 PM, Mel Gorman wrote: The number of pages kswapd can reclaim is bound by the number of pages it scans which is related to the size of the zone and the

Re: [PATCH 03/10] mm: vmscan: Flatten kswapd priority loop

2013-03-18 Thread Simon Jeons
Hi Mel, On 03/17/2013 09:04 PM, Mel Gorman wrote: kswapd stops raising the scanning priority when at least SWAP_CLUSTER_MAX pages have been reclaimed or the pgdat is considered balanced. It then rechecks if it needs to restart at DEF_PRIORITY and whether high-order reclaim needs to be reset. This

Re: [PATCH 01/10] mm: vmscan: Limit the number of pages kswapd reclaims at each priority

2013-03-18 Thread Simon Jeons
Hi Mel, On 03/17/2013 09:04 PM, Mel Gorman wrote: The number of pages kswapd can reclaim is bound by the number of pages it scans which is related to the size of the zone and the scanning priority. In many cases the priority remains low because it's reset every SWAP_CLUSTER_MAX reclaimed pages bu

Re: [PATCH 03/10] mm: vmscan: Flatten kswapd priority loop

2013-03-18 Thread Simon Jeons
Hi Mel, On 03/17/2013 09:04 PM, Mel Gorman wrote: kswapd stops raising the scanning priority when at least SWAP_CLUSTER_MAX pages have been reclaimed or the pgdat is considered balanced. It then rechecks if it needs to restart at DEF_PRIORITY and whether high-order reclaim needs to be reset. This

Re: [PATCH 07/10] mm: vmscan: Block kswapd if it is encountering pages under writeback

2013-03-18 Thread Simon Jeons
On 03/17/2013 09:04 PM, Mel Gorman wrote: Historically, kswapd used to congestion_wait() at higher priorities if it was not making forward progress. This made no sense as the failure to make progress could be completely independent of IO. It was later replaced by wait_iff_congested() and removed

Re: [PATCHv2, RFC 00/30] Transparent huge page cache

2013-03-18 Thread Simon Jeons
Hi Kirill, On 03/18/2013 07:19 PM, Kirill A. Shutemov wrote: Simon Jeons wrote: On 03/18/2013 12:03 PM, Simon Jeons wrote: Hi Kirill, On 03/15/2013 01:50 AM, Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" Here's the second version of the patchset. The intend of the w

Re: [PATCH, RFC 00/16] Transparent huge page cache

2013-03-18 Thread Simon Jeons
On 01/28/2013 05:24 PM, Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" Here's first steps towards huge pages in page cache. The intend of the work is get code ready to enable transparent huge page cache for the most simple fs -- ramfs. It's not yet near feature-complete. It only provide

Re: [PATCHv2, RFC 00/30] Transparent huge page cache

2013-03-17 Thread Simon Jeons
Hi Kirill, On 03/15/2013 01:50 AM, Kirill A. Shutemov wrote: From: "Kirill A. Shutemov" Here's the second version of the patchset. The intend of the work is get code ready to enable transparent huge page cache for the most simple fs -- ramfs. We have read()/write()/mmap() functionality now. S

Re: Inactive memory keep growing and how to release it?

2013-03-15 Thread Simon Jeons
On 03/14/2013 08:39 PM, Hillf Danton wrote: On Sat, Mar 9, 2013 at 10:14 AM, Will Huck wrote: Cc experts. Hugh, Johannes, On 03/04/2013 08:21 PM, Lenky Gao wrote: 2013/3/4 Zlatko Calusic : The drop_caches mechanism doesn't free dirty page cache pages. And your bash script is creating a lot o

Re: Inactive memory keep growing and how to release it?

2013-03-15 Thread Simon Jeons
On 03/14/2013 06:14 PM, Michal Hocko wrote: On Mon 04-03-13 17:52:22, Lenky Gao wrote: Hi, When i just run a test on Centos 6.2 as follows: #!/bin/bash while true do file="/tmp/filetest" echo $file dd if=/dev/zero of=${file} bs=512 count=204800 &> /dev/null

Re: [PATCH] add extra free kbytes tunable

2013-03-08 Thread Simon Jeons
Hi Hugh, On 03/02/2013 11:08 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is

Re: [PATCH v4 001/002] mm: limit growth of 3% hardcoded other user reserve

2013-03-05 Thread Simon Jeons
On 03/06/2013 07:38 AM, Andrew Shewmaker wrote: Limit the growth of the memory reserved for other processes to the smaller of 3% or 8MB. This affects only OVERCOMMIT_NEVER. Signed-off-by: Andrew Shewmaker Please add changelog, otherwise it's for other guys to review. --- Rebased onto v3.

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 11:08 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 09:42 AM, Hugh Dickins wrote: On Sat, 2 Mar 2013, Simon Jeons wrote: In function __add_to_swap_cache if add to radix tree successfully will result in increase NR_FILE_PAGES, why? This is anonymous page instead of file backed page. Right, that's hard to understand wi

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/02/2013 06:33 AM, Hugh Dickins wrote: On Fri, 1 Mar 2013, Simon Jeons wrote: On 03/01/2013 05:22 PM, Simon Jeons wrote: On 02/23/2013 01:56 AM, Johannes Weiner wrote: Mapped file pages have to get scanned twice before they are reclaimed because we don't have enough usage inform

Re: [PATCH] x86: mm: Check if PUD is large when validating a kernel address v2

2013-03-01 Thread Simon Jeons
On 03/01/2013 05:35 PM, Chen Gong wrote: On Fri, Mar 01, 2013 at 05:21:35PM +0800, Simon Jeons wrote: Date: Fri, 01 Mar 2013 17:21:35 +0800 From: Simon Jeons To: Mel Gorman , Ingo Molnar , Andrew Morton , linux-kernel@vger.kernel.org, linux...@kvack.org, r...@redhat.com, mho...@suse.cz

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
On 03/01/2013 05:22 PM, Simon Jeons wrote: Hi Johannes, On 02/23/2013 01:56 AM, Johannes Weiner wrote: On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the

Re: [PATCH] add extra free kbytes tunable

2013-03-01 Thread Simon Jeons
Hi Johannes, On 02/23/2013 01:56 AM, Johannes Weiner wrote: On Tue, Feb 19, 2013 at 09:19:27PM -0800, dormando wrote: The problem is that adding this tunable will constrain future VM implementations. We will forever need to at least retain the pseudo-file. We will also need to make some effor

Re: [PATCH] x86: mm: Check if PUD is large when validating a kernel address v2

2013-03-01 Thread Simon Jeons
On 03/01/2013 05:15 PM, Chen Gong wrote: On Fri, Mar 01, 2013 at 02:43:53PM +0800, Simon Jeons wrote: Date: Fri, 01 Mar 2013 14:43:53 +0800 From: Simon Jeons To: Mel Gorman CC: Ingo Molnar , Andrew Morton , linux-kernel@vger.kernel.org, linux...@kvack.org, r...@redhat.com, mho...@suse.cz

Re: [PATCH] x86: mm: Check if PUD is large when validating a kernel address v2

2013-02-28 Thread Simon Jeons
On 02/13/2013 07:02 PM, Mel Gorman wrote: Andrew or Ingo, please pick up. Changelog since v1 o Add reviewed-bys and acked-bys A user reported a bug whereby a backup process accessing /proc/kcore caused an oops. BUG: unable to handle kernel paging request at bb00ff33b000 IP: [] kern_

Re: [RFC PATCH v2 2/2] mm: tuning hardcoded reserved memory

2013-02-28 Thread Simon Jeons
On 02/28/2013 05:09 AM, Andrew Shewmaker wrote: Add a rootuser_reserve_pages knob to allow admins of large memory systems running with overcommit disabled to change the hardcoded memory reserve to something other than 3%. Signed-off-by: Andrew Shewmaker --- Patch based off of mmotm git tree a

Re: [PATCH 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-20 Thread Simon Jeons
On 02/20/2013 06:23 PM, Lin Feng wrote: Hi Simon, On 02/20/2013 05:58 PM, Simon Jeons wrote: The other is that this almost certainly broken for transhuge page handling. gup returns the head and tail pages and ordinarily this is ok When need gup thp? in kvm case? gup just pins the wanted

Re: [PATCH 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-20 Thread Simon Jeons
On 02/05/2013 09:32 PM, Mel Gorman wrote: On Tue, Feb 05, 2013 at 11:57:22AM +, Mel Gorman wrote: + migrate_pre_flag = 1; + } + + if (!isolate_lru_page(pages[i])) { + inc_zone_page_state(p

Re: [PATCH] mm: cma: fix accounting of CMA pages placed in high memory

2013-02-19 Thread Simon Jeons
On 02/20/2013 10:57 AM, Kyungmin Park wrote: On Tue, Feb 19, 2013 at 10:27 PM, Simon Jeons wrote: On 02/05/2013 03:10 PM, Marek Szyprowski wrote: Hello, On 2/5/2013 12:34 AM, Minchan Kim wrote: On Mon, Feb 04, 2013 at 11:27:05AM +0100, Marek Szyprowski wrote: The total number of low memory

Re: [PATCH] mm: cma: fix accounting of CMA pages placed in high memory

2013-02-19 Thread Simon Jeons
On 02/05/2013 03:10 PM, Marek Szyprowski wrote: Hello, On 2/5/2013 12:34 AM, Minchan Kim wrote: On Mon, Feb 04, 2013 at 11:27:05AM +0100, Marek Szyprowski wrote: > The total number of low memory pages is determined as > totalram_pages - totalhigh_pages, so without this patch all CMA > pageblock

Re: PAGE_CACHE_SIZE vs. PAGE_SIZE

2013-02-19 Thread Simon Jeons
On 01/18/2013 11:57 PM, Kirill A. Shutemov wrote: Hi, PAGE_CACHE_* macros were introduced long time ago in hope to implement page cache with larger chunks than one page in future. In fact it was never done. Some code paths assume PAGE_CACHE_SIZE <= PAGE_SIZE. E.g. we use zero_user_segments() t

Re: [patch 1/2] mm: fincore()

2013-02-19 Thread Simon Jeons
Hi Johannes, On 02/15/2013 02:34 PM, Johannes Weiner wrote: On Mon, Feb 11, 2013 at 02:12:39PM -0800, Andrew Morton wrote: Also, having to mmap the file to be able to query pagecache state is a hack. Whatever happened to the fincore() patch? I don't know, but how about this one: --- From: Joh

Re: [PATCH] zsmalloc: Fix TLB coherency and build problem

2013-02-19 Thread Simon Jeons
On 01/28/2013 09:00 AM, Minchan Kim wrote: Recently, Matt Sealey reported he fail to build zsmalloc caused by using of local_flush_tlb_kernel_range which are architecture dependent function so !CONFIG_SMP in ARM couldn't implement it so it ends up build error following as. Confuse me! 1) Why I

Re: [RFC] Reproducible OOM with just a few sleeps

2013-02-17 Thread Simon Jeons
On 01/14/2013 11:00 PM, Dave Hansen wrote: On 01/11/2013 07:31 PM, paul.sz...@sydney.edu.au wrote: Seems that any i386 PAE machine will go OOM just by running a few processes. To reproduce: sh -c 'n=0; while [ $n -lt 1 ]; do sleep 600 & ((n=n+1)); done' My machine has 64GB RAM. With previ

Re: [PATCH] zsmalloc: Add Kconfig for enabling PTE method

2013-02-03 Thread Simon Jeons
On Mon, 2013-02-04 at 09:23 +0900, Minchan Kim wrote: > Zsmalloc has two methods 1) copy-based and 2) pte based to access > allocations that span two pages. > You can see history why we supported two approach from [1]. > > But it was bad choice that adding hard coding to select architecture > whic

Re: [PATCHv4 0/7] zswap: compressed swap caching

2013-02-03 Thread Simon Jeons
On Fri, 2013-02-01 at 09:13 -0600, Seth Jennings wrote: > On 01/31/2013 07:39 PM, Simon Jeons wrote: > > Hi Seth, > > On Tue, 2013-01-29 at 15:40 -0600, Seth Jennings wrote: > > >> Performance, Kernel Building: > >> > >> Setup > >>

  1   2   >