Re: problems with mmap() and disk caching

2012-05-22 Thread Andrey Zonov
On 4/30/12 3:49 AM, Alan Cox wrote: On 04/11/2012 01:07, Andrey Zonov wrote: On 10.04.2012 20:19, Alan Cox wrote: On 04/09/2012 10:26, John Baldwin wrote: On Thursday, April 05, 2012 11:54:31 am Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53P

Re: problems with mmap() and disk caching

2012-04-29 Thread Alan Cox
On 04/11/2012 01:07, Andrey Zonov wrote: On 10.04.2012 20:19, Alan Cox wrote: On 04/09/2012 10:26, John Baldwin wrote: On Thursday, April 05, 2012 11:54:31 am Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I

Re: problems with mmap() and disk caching

2012-04-10 Thread Andrey Zonov
On 10.04.2012 20:19, Alan Cox wrote: On 04/09/2012 10:26, John Baldwin wrote: On Thursday, April 05, 2012 11:54:31 am Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the who

mlock/mlockall (was: Re: problems with mmap() and disk caching)

2012-04-10 Thread Dieter BSD
Andrey writes: > Wired memory: kernel memory and yes, application may get wired memory > through mlock()/mlockall(), but I haven't seen any real application > which calls mlock(). Apps with real time considerations may need to lock memory to prevent having to wait for page/swap. __

Re: problems with mmap() and disk caching

2012-04-10 Thread Alan Cox
On 04/09/2012 10:26, John Baldwin wrote: On Thursday, April 05, 2012 11:54:31 am Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer, then I work w

Re: problems with mmap() and disk caching

2012-04-09 Thread John Baldwin
On Thursday, April 05, 2012 11:54:31 am Alan Cox wrote: > On 04/04/2012 02:17, Konstantin Belousov wrote: > > On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: > >> Hi, > >> > >> I open the file, then call mmap() on the whole file and get pointer, > >> then I work with this pointer. I

Re: problems with mmap() and disk caching

2012-04-09 Thread Konstantin Belousov
On Mon, Apr 09, 2012 at 03:35:30PM +0400, Andrey Zonov wrote: > On Mon, Apr 9, 2012 at 1:18 PM, Konstantin Belousov > wrote: > > On Mon, Apr 09, 2012 at 11:17:41AM +0400, Andrey Zonov wrote: > >> On 06.04.2012 12:13, Konstantin Belousov wrote: > >> >On Thu, Apr 05, 2012 at 11:54:53PM +0400, Andre

Re: problems with mmap() and disk caching

2012-04-09 Thread Andrey Zonov
On Mon, Apr 9, 2012 at 1:18 PM, Konstantin Belousov wrote: > On Mon, Apr 09, 2012 at 11:17:41AM +0400, Andrey Zonov wrote: >> On 06.04.2012 12:13, Konstantin Belousov wrote: >> >On Thu, Apr 05, 2012 at 11:54:53PM +0400, Andrey Zonov wrote: [snip] >> >>I always thought that active memory this is a

Re: problems with mmap() and disk caching

2012-04-09 Thread Konstantin Belousov
On Mon, Apr 09, 2012 at 11:17:41AM +0400, Andrey Zonov wrote: > On 06.04.2012 12:13, Konstantin Belousov wrote: > >On Thu, Apr 05, 2012 at 11:54:53PM +0400, Andrey Zonov wrote: > >>On 05.04.2012 23:41, Konstantin Belousov wrote: > >>>On Thu, Apr 05, 2012 at 11:33:46PM +0400, Andrey Zonov wrote: > >

Re: problems with mmap() and disk caching

2012-04-09 Thread Andrey Zonov
On 06.04.2012 12:13, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 11:54:53PM +0400, Andrey Zonov wrote: On 05.04.2012 23:41, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 11:33:46PM +0400, Andrey Zonov wrote: On 05.04.2012 19:54, Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belo

Re: problems with mmap() and disk caching

2012-04-06 Thread Alan Cox
On 04/06/2012 03:38, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 01:25:49PM -0500, Alan Cox wrote: On 04/05/2012 12:31, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 10:54:31AM -0500, Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53P

Re: problems with mmap() and disk caching

2012-04-06 Thread Alan Cox
On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer, then I work with this pointer. I expect that page should be only once touched to get it into the memory (disk c

Re: problems with mmap() and disk caching

2012-04-06 Thread Konstantin Belousov
On Thu, Apr 05, 2012 at 01:25:49PM -0500, Alan Cox wrote: > On 04/05/2012 12:31, Konstantin Belousov wrote: > >On Thu, Apr 05, 2012 at 10:54:31AM -0500, Alan Cox wrote: > >>On 04/04/2012 02:17, Konstantin Belousov wrote: > >>>On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: > Hi, >

Re: problems with mmap() and disk caching

2012-04-06 Thread Konstantin Belousov
On Thu, Apr 05, 2012 at 11:54:53PM +0400, Andrey Zonov wrote: > On 05.04.2012 23:41, Konstantin Belousov wrote: > >On Thu, Apr 05, 2012 at 11:33:46PM +0400, Andrey Zonov wrote: > >>On 05.04.2012 19:54, Alan Cox wrote: > >>>On 04/04/2012 02:17, Konstantin Belousov wrote: > On Tue, Apr 03, 2012 a

Re: problems with mmap() and disk caching

2012-04-05 Thread Andrey Zonov
On 05.04.2012 23:54, Andrey Zonov wrote: On 05.04.2012 23:41, Konstantin Belousov wrote: You do use UFS, right ? Yes. I've run test on ZFS. Mem: 2645M Active, 363M Inact, 2042M Wired, 1406M Buf, 42G Free $ ./mmap /mnt/random Mem: 3669M Active, 363M Inact, 3067M Wired, 1406M Buf, 40G Free

Re: problems with mmap() and disk caching

2012-04-05 Thread Andrey Zonov
On 05.04.2012 23:41, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 11:33:46PM +0400, Andrey Zonov wrote: On 05.04.2012 19:54, Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: [snip] This is what I expect. But w

Re: problems with mmap() and disk caching

2012-04-05 Thread Konstantin Belousov
On Thu, Apr 05, 2012 at 11:33:46PM +0400, Andrey Zonov wrote: > On 05.04.2012 19:54, Alan Cox wrote: > >On 04/04/2012 02:17, Konstantin Belousov wrote: > >>On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: > [snip] > >>>This is what I expect. But why this doesn't work without reading fi

Re: problems with mmap() and disk caching

2012-04-05 Thread Andrey Zonov
On 05.04.2012 19:54, Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: [snip] This is what I expect. But why this doesn't work without reading file manually? Issue seems to be in some change of the behaviour of the re

Re: problems with mmap() and disk caching

2012-04-05 Thread Alan Cox
On 04/05/2012 12:31, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 10:54:31AM -0500, Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer, then

Re: problems with mmap() and disk caching

2012-04-05 Thread Konstantin Belousov
On Thu, Apr 05, 2012 at 10:54:31AM -0500, Alan Cox wrote: > On 04/04/2012 02:17, Konstantin Belousov wrote: > >On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: > >>Hi, > >> > >>I open the file, then call mmap() on the whole file and get pointer, > >>then I work with this pointer. I ex

Re: problems with mmap() and disk caching

2012-04-05 Thread Alan Cox
On 04/04/2012 04:36, Andrey Zonov wrote: On 04.04.2012 11:17, Konstantin Belousov wrote: Calling madvise(MADV_RANDOM) fixes the issue, because the code to deactivate/cache the pages is turned off. On the other hand, it also turns of read-ahead for faulting, and the first loop becomes eternally

Re: problems with mmap() and disk caching

2012-04-05 Thread Alan Cox
On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer, then I work with this pointer. I expect that page should be only once touched to get it into the memory (disk c

Re: problems with mmap() and disk caching

2012-04-04 Thread Andrey Zonov
I forgot to attach my test program. On 04.04.2012 13:36, Andrey Zonov wrote: On 04.04.2012 11:17, Konstantin Belousov wrote: Calling madvise(MADV_RANDOM) fixes the issue, because the code to deactivate/cache the pages is turned off. On the other hand, it also turns of read-ahead for faulting,

Re: problems with mmap() and disk caching

2012-04-04 Thread Andrey Zonov
On 04.04.2012 11:17, Konstantin Belousov wrote: Calling madvise(MADV_RANDOM) fixes the issue, because the code to deactivate/cache the pages is turned off. On the other hand, it also turns of read-ahead for faulting, and the first loop becomes eternally long. Now it takes 5 times longer. Anyw

Re: problems with mmap() and disk caching

2012-04-04 Thread Konstantin Belousov
On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: > Hi, > > I open the file, then call mmap() on the whole file and get pointer, > then I work with this pointer. I expect that page should be only once > touched to get it into the memory (disk cache?), but this doesn't work! > > I w

problems with mmap() and disk caching

2012-04-03 Thread Andrey Zonov
Hi, I open the file, then call mmap() on the whole file and get pointer, then I work with this pointer. I expect that page should be only once touched to get it into the memory (disk cache?), but this doesn't work! I wrote the test (attached) and ran it for the 1G file generated from /dev/r