George Sakkis wrote:
> It's around 400MB.
On Windows you may not be able to map a file of this size into memory
because of virtual address space fragmentation. A Win32 process has
only 2G of virtual address space, and DLLs tend to get scattered
through out that address space.
> As I said, I cann
Nick Craig-Wood schrieb:
> So presumably it is python generating a MemoryError. It is asking for
> a new bit of memory and it is failing so it throws a MemoryError.
>
> Could memory allocation under windows be affected by a large chunk of
> mmap()ed file which is physically swapped in at the time
Laszlo Nagy schrieb:
>
>> In fact, memory that is read in because of mmap should *never* cause
>> a MemoryError.
> This is certainly not true. You can run out of virtual address space by
> reading data from a memory mapped file.
That is true, but not what I said. I said you cannot run out of mem
> It's around 400MB. As I said, I cannot reproduce the MemoryError
> locally since I have 1GB physical space but IIRC the user who reported
> it had less. Actually I am less concerned about whether a MemoryError
> is raised or not in this case and more about the fact that even if
> there's no exce
> In fact, memory that is read in because of mmap should *never* cause
> a MemoryError.
This is certainly not true. You can run out of virtual address space by
reading data from a memory mapped file.
> Python calls MapViewOfFile when mmap.mmap is invoked,
> at which point the operating commits t
Dennis Lee Bieber wrote:
> On 21 Jan 2007 13:32:19 -0800, "George Sakkis" <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
>
> >
> > The file is written once and then opened as read-only, there's no
> > flushing. So if caching is completely up to the OS, I take it that my
>
>
George Sakkis <[EMAIL PROTECTED]> wrote:
> The file is written once and then opened as read-only, there's no
> flushing. So if caching is completely up to the OS, I take it that my
> options are either (1) modify my algorithms so that they work in
> fixed-size batches instead of arbitrarily lon
Martin v. Löwis <[EMAIL PROTECTED]> wrote:
> In fact, memory that is read in because of mmap should *never* cause
> a MemoryError. Python calls MapViewOfFile when mmap.mmap is invoked,
> at which point the operating commits to providing that much address
> space to the application, along with b
George Sakkis schrieb:
>> You must be misinterpreting what you are seeing. It's the operating
>> system that decides what part of a memory-mapped file are held in
>> memory, and that is certainly not without limits.
>
> Sure; what I meant was that that whatever the limit is, it's high
> enough tha
Martin v. Löwis wrote:
> George Sakkis schrieb:
> > I've been trying to track down a memory leak (which I initially
> > attributed erroneously to numpy) and it turns out to be caused by a
> > memory mapped file. It seems that mmap caches without limit the chunks
> > it reads, as the memory usage g
George Sakkis schrieb:
> I've been trying to track down a memory leak (which I initially
> attributed erroneously to numpy) and it turns out to be caused by a
> memory mapped file. It seems that mmap caches without limit the chunks
> it reads, as the memory usage grows to several hundreds MBs accor
Nick Craig-Wood wrote:
> George Sakkis <[EMAIL PROTECTED]> wrote:
> > I've been trying to track down a memory leak (which I initially
> > attributed erroneously to numpy) and it turns out to be caused by a
> > memory mapped file. It seems that mmap caches without limit the chunks
> > it reads,
George Sakkis <[EMAIL PROTECTED]> wrote:
> I've been trying to track down a memory leak (which I initially
> attributed erroneously to numpy) and it turns out to be caused by a
> memory mapped file. It seems that mmap caches without limit the chunks
> it reads, as the memory usage grows to seve
13 matches
Mail list logo
