On Mon, 21 Mar 2011 10:24:22 +0000, Greg Stark wrote:
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus <j...@agliodbs.com>
wrote:
To take the opposite approach... has anyone looked at having the OS
just manage all caching for us? Something like MMAPed shared buffers?
Even if we find the issue with large shared buffers, we still can't
dedicate serious amounts of memory to them because of work_mem
issues. Granted, that's something else on the TODO list, but it
really seems like we're re-inventing the wheels that the OS has
already created here...
A lot of people have talked about it. You can find references to mmap
going at least as far back as 2001 or so. The problem is that it
would
depend on the OS implementing things in a certain way and
guaranteeing
things we don't think can be portably assumed. We would need to mlock
large amounts of address space which most OS's don't allow, and we
would need to at least mlock and munlock lots of small bits of memory
all over the place which would create lots and lots of mappings which
the kernel and hardware implementations would generally not
appreciate.
Actually, just from curious, I done test with mmap, and I got 2% boost
on data reading, maybe because of skipping memcpy in fread. I really
curious how fast, if even, it will be if I add some good and needed
stuff and how e.g. vacuum will work.
<snip>
2-level caches work well for a variety of applications.
I think 2-level caches with simple heuristics like "pin all the
indexes" is unlikely to be helpful. At least it won't optimize the
average case and I think that's been proven. It might be helpful for
optimizing the worst-case which would reduce the standard deviation.
Perhaps we're at the point now where that matters.
Actually, 2nd level caches do not pin index buffer. It's just, in
simple words, some set of reserved buffers' ids to be used for index
pages, all logic with pining, etc. it's same, the difference is that
default level operation will not touch 2nd level. I post some reports
from my simple tests. When I was experimenting with 2nd level caches I
saw that some operations may swap out system tables buffers, too.
<snip>
Regards,
Radek
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
