Hi Larry, >From my understanding this is correct, except the part about the code history: I am not sure what PHP did in the past (before the new streams API), so before saying that it was always this way, you should review ext/std/file.c and main/streams.c in the GIT/SVN/CVS history! But the other notes seem to be the reason for the "persistent urban legends" :-)
Uwe P.S.: By the way, I will have a MMap blog post, too; focusing on the same urban legends about to the use of Lucene's MMapDirectory in Apache Lucene and Apache Solr. I am just a little bit overcrowded with work. ----- Uwe Schindler theta...@php.net - http://www.php.net NSAPI SAPI developer Bremen, Germany > -----Original Message----- > From: Larry Garfield [mailto:la...@garfieldtech.com] > Sent: Monday, April 30, 2012 7:16 PM > To: internals@lists.php.net > Subject: Re: [PHP-DEV] readfile() memory usage > > Thanks all. So it sounds like the answer is: > > 1) readfile() has always been memory-safe as far as PHP is concerned. > 2) Because it uses mmap(), GFL trying to understand its memory usage from > top. > 3) Operating systems have gotten better at such things in the past decade. > 4) So given #2 and #3, the "readfile() will kill your memory, don't use it" line is > a persistent urban legend that belongs on Snopes as debunked. Looping on > fread() for performance is a red herring. > > Is that an accurate summary? If so, I will blog my benchmark results and this > conversation. > > --Larry Garfield > > On 4/30/12 5:33 AM, Sherif Ramadan wrote: > >> Mmapping of course uses memory, but the memory used here is not from > >> PHP's memory manager, it's memory that's already used for the O/S > >> cache. The memory mapping used here does not enforce loading the file > >> contents into O/S cache; it just gets a virtual address into the O/S > >> cache. If the actual file contents are not yet in O/S cache, the O/S > >> will hit a page fault and load the pages into memory. Apache Server uses > the same mechanism to serve files. > >> > >> Maybe the "user confusion" about memory usage comes from that fact > >> (they see lots of *virtual* memory used by PHP when viewed in top). I > >> know this user confusion from my work in the Apache Lucene/Solr > >> project, where one option (used on 64 bit operating systems) is to > >> memory-map the whole Lucene full-text index. Users then see hundreds of > Gigabytes of "virtual memory" > >> usage in TOP / Windows Task Manager and are afraid of running their > >> machine out of memory. This is always hard to explain to people that > >> are not used to the term "virtual memory". > >> > > > > > > That's a very good point and a detailed look at the stack can show > > some of the underlying mechanics coming from readfile and how its > > pretty much is just implemented like fpassthru delegating a stream, as > > you said. > > > > http://i.imgur.com/PWiTv.png > > > > > >> > >>> About php memory usage, one has to use an external tools to actually > >>> see > >> this > >>> memory usage as it is not managed by the zend memory manager. > >> > >> Of course... > >> > > > > Also, running valgrind and taking a closer look at what memory blocks > > PHP is allocating here it can be better determined what's leaking and > > what isn't, of course... > > > > http://pastebin.com/LSQcUsBL > > > > > > I can certainly attest to users being deceived by memory readings in > > top in the past. It can be very deceiving if you think look at what > > free memory top or `free` shows you especially after some huge > > allocation. There's a clear difference here though between the Zend > > memory manager allocating these blocks in PHP and what's going on with > > readfile. The memory the system knows is available to it isn't being > > tied up in this case. > > > > -- > PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: > http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
