Re: mmap() question

2013-10-11 Thread Dmitry Sivachenko 

On 11.10.2013, at 9:17, Konstantin Belousov  wrote:

> On Wed, Oct 09, 2013 at 03:42:27PM +0400, Dmitry Sivachenko wrote:
>> Hello!
>> 
>> I have a program which mmap()s a lot of large files (total size more that 
>> RAM and I have no swap), but it needs only small parts of that files at a 
>> time.
>> 
>> My understanding is that when using mmap when I access some memory region OS 
>> reads the relevant portion of that file from disk and caches the result in 
>> memory.  If there is no free memory, OS will purge previously read part of 
>> mmap'ed file to free memory for the new chunk.
>> 
>> But this is not the case.  I use the following simple program which gets 
>> list of files as command line arguments, mmap()s them all and then selects 
>> random file and random 1K parts of that file and computes a XOR of bytes 
>> from that region.
>> After some time the program dies:
>> pid 63251 (a.out), uid 1232, was killed: out of swap space
>> 
>> It seems I incorrectly understand how mmap() works, can you please clarify 
>> what's going wrong?
>> 
>> I expect that program to run indefinitely, purging some regions out of RAM 
>> and reading the relevant parts of files.
>> 
> 
> You did not specified several very important parameters for your test:
> 1. total amount of RAM installed


24GB


> 2. count of the test files and size of the files

To be precise: I used 57 files with size varied form 74MB to 19GB.
The total size of these files is 270GB.

> 3. which filesystem files are located at


UFS @ SSD drive

> 4. version of the system.


FreeBSD 9.2-PRERELEASE #0 r254880M: Wed Aug 28 11:07:54 MSK 2013
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: What's the state of AF-4Kn support?

2013-10-11 Thread Ravi Pokala 

Yes, the BIOS calls have always only used 512 byte sectors. There would
have to be an updated spec for those, and it would be a bit of a PITA to
use. I suspect the "right" answer for this on x86 is UEFI.

Yeah, that's the conclusion I reached as well. Though it occurs to me, aren't 
we already part-way there w/ our IA64 support? EFI originated w/ Itanium and 
was required to boot those systems, so shouldn't we be able to leverage much of 
that work for (U)EFI on amd64?

In any case, the primary motivator (at least, for me) is being able to boot 
from AF-4Kn drives; based on the most recent roadmaps I've seen, the enterprise 
HDD vendors are committing to support AF-512e for a good while longer, so it's 
not as urgent as I thought it was when I opened this thread a few weeks ago.

Thanks,

--rp

On Oct 10, 2013, at 01:57 PM, John Baldwin  wrote:

On Monday, September 23, 2013 10:58:19 am Ravi Pokala wrote:
-Original Message-
From: Jia-Shiun Li 
Date: Sunday, September 22, 2013 11:22 PM
To: Ravi Pokala 
Cc: "freebsd-hardw...@freebsd.org" ,

Subject: Re: What's the state of AF-4Kn support?

On Wed, Sep 18, 2013 at 10:49 PM, Ravi Pokala  wrote:


...


CC -hackers.

Thanks for the clarification. Is there any 4Kn HDDs shopping now? I am
not aware of any.

Good question. I had the impression that some currently shipping drives
were AF-4Kn, but spot-checking some of the drives listed in
src/cam/ata/ata_da.c::ada_quirk_table[]
against their datasheets, suggests that they're AF-512e. So, their being
flagged w/ ADA_Q_4K is "just" a performance optimization.

BTW I believe UFS and ZFS have proper design for 4K-sectors, but FreeBSD
needs some ecosystem connections to get samples early to test,
incorporate supports and validate for it. Or we will need to wait until
it appears on market and someone got caught into some kind of bugs.

Yeah, based on my reading of the code, it looks like the ATACAM layer and
higher (GEOM, filesystems) take the physical block size into account. That
just leaves the bootstrap code. Now that I've taken a second look, it
seems as though at least 'pmbr' only works in terms of 512 bytes. :-(

Yes, the BIOS calls have always only used 512 byte sectors. There would
have to be an updated spec for those, and it would be a bit of a PITA to
use. I suspect the "right" answer for this on x86 is UEFI.

--
John Baldwin
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"