Hi ladies and gentlemen,
Please help - I am stuck.
Looks like memory pages go unaccounted for, somewhere in the system,
when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu
distribution).
So what I do is I have two simple applications that allocate 1GB
memory each and then proceed to write arbitrary data to the allocated
memory, with page granularity.
The system has a total of 1.3GB RAM to simulate memory pressure etc
and encourage swapping.
To test the basic functionality of my approach, I included a simple
"next-page" prefetch code into the mm/memory.c, just after the call to
do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637)
What it should do is prefetch 10 pages following the one that has just
been swapped in on-demand. I don't really care which ones, what
matters is want to try something different than reading sequentially
from the swap file (read-ahead).
See the code below.
The problem is, if I run the test apps for the sufficient amount of
time, the "used" memory increases to the point where the applications
are killed by an OOM-killer.
After that, the available system memory is about half of what it
should have been (e.g. 600MB/1.3GB). Same thing happens to memory
usage if I stop the applications before the OOM-killer is triggered,
I am also attaching the /proc/meminfo contents for comparison on a
freshly-booted system vs. one where the two apps have been running for
200 seconds before quitting, with and without the prefetch code in
kernel.
Note how Active and Inactive show a lot of memory being "used" for something.
The printouts were made when the system is completely idle with
nothing running etc.
I also tried prefetching from within the swapin_readahead() function
(http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with
the same result - memory gets "used up".
I would greatly appreciate any help!
Pretty much ran out of ideas as to what could be causing this.
Thanks a lot,
Slava
============== code fragment used for prefetching ============
(( at the beginning of handle_pte_fault() function...
int rett, ii;
pte_t *ptee;
swp_entry_t sentry;
struct page *page;
))
rett = do_swap_page(mm, vma, address,
pte, pmd, flags, entry);
////////////////////////////////////////////////////////////////
/// test for memory leaks////////////////////
for(ii=1;ii<10;ii++){
ptee = pte_offset_map(pmd, address + (ii*2)<<12);
if(pte_present(*ptee) || pte_none(*ptee) ||
pte_file(*ptee)) return rett; // no such page in page table
sentry = pte_to_swp_entry(*ptee);
if (non_swap_entry(sentry)){
// printk(KERN_ALERT " non swap entry\n");
return rett;
}
delayacct_set_flag(DELAYACCT_PF_SWAPIN);
page = lookup_swap_cache(sentry);
if (!page) {
page = read_swap_cache_async(sentry,
GFP_HIGHUSER_MOVABLE, vma, address);
if(page) page_cache_release(page);
}
lru_add_drain();
// if(page) unlock_page(page);
delayacct_clear_flag(DELAYACCT_PF_SWAPIN);
}
// printk(KERN_ALERT "swapped pages\n");
////////////////////////////////////////////
//////////////////////////////////////////
return rett;
===================== end code fragment =====================
prefetch: 200s run:
MemTotal: 1270536 kB
MemFree: 668748 kB
Buffers: 5304 kB
Cached: 33248 kB
SwapCached: 120424 kB
Active: 336016 kB
Inactive: 221792 kB
Active(anon): 328280 kB
Inactive(anon): 190984 kB
Active(file): 7736 kB
Inactive(file): 30808 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 4191228 kB
SwapFree: 4052040 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 6008 kB
Mapped: 4596 kB
Shmem: 8 kB
Slab: 23656 kB
SReclaimable: 10304 kB
SUnreclaim: 13352 kB
KernelStack: 800 kB
PageTables: 2008 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 4826496 kB
Committed_AS: 72232 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 267280 kB
VmallocChunk: 34359469179 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 34816 kB
DirectMap2M: 1378304 kB
fresh boot:
MemTotal: 1270536 kB
MemFree: 1118524 kB
Buffers: 13236 kB
Cached: 79364 kB
SwapCached: 0 kB
Active: 37896 kB
Inactive: 71008 kB
Active(anon): 16336 kB
Inactive(anon): 252 kB
Active(file): 21560 kB
Inactive(file): 70756 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 4191228 kB
SwapFree: 4191228 kB
Dirty: 24 kB
Writeback: 0 kB
AnonPages: 16212 kB
Mapped: 7320 kB
Shmem: 280 kB
Slab: 23496 kB
SReclaimable: 11060 kB
SUnreclaim: 12436 kB
KernelStack: 856 kB
PageTables: 1656 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 4826496 kB
Committed_AS: 60072 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 267280 kB
VmallocChunk: 34359470199 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 36864 kB
DirectMap2M: 1376256 kB
no prefetch: 200s run:
MemTotal: 1270536 kB
MemFree: 1219856 kB
Buffers: 344 kB
Cached: 4348 kB
SwapCached: 1512 kB
Active: 1768 kB
Inactive: 6284 kB
Active(anon): 172 kB
Inactive(anon): 3124 kB
Active(file): 1596 kB
Inactive(file): 3160 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 4191228 kB
SwapFree: 4168800 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 3340 kB
Mapped: 1948 kB
Shmem: 0 kB
Slab: 22416 kB
SReclaimable: 9580 kB
SUnreclaim: 12836 kB
KernelStack: 768 kB
PageTables: 1972 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 4826496 kB
Committed_AS: 71036 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 267280 kB
VmallocChunk: 34359470199 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 36864 kB
DirectMap2M: 1376256 kB
#include <stdio.h>
#include <stdlib.h>
int main(){
unsigned long i=2;
char *mem = (char*)malloc(1024*1024*1424); // 1 gig
if(mem)
while(1){
mem[i] = rand();
i+=4096;
i = i%(1024*1024*1024);
}
return 0;
}
