On Sun, Dec 19, 2010 at 6:14 AM, Martijn van Oosterhout <klep...@svana.org> wrote: > On Sat, Dec 18, 2010 at 11:59:33PM -0800, Jeff Janes wrote: >> On Sat, Dec 18, 2010 at 10:11 PM, flyusa2010 fly <flyusa2...@gmail.com> >> wrote: >> > hi, folks! >> > I see that shared cache is implemented by system v shared memory. I wonder >> > whether data in this area can be swapped out to disk. >> > Isn't it bad that we read data from disk, put data in shared cache, and >> > finally data in shared cache is swapped to disk again! >> > Why not use shmctl(..SHM_LOCK..) to pin data in main memory? >> > Thanks! >> >> I've tried that on a recent linux kernel, to see if it would allow >> shared_buffers to usefully be a large fraction of total memory. It >> didn't help. So either swapping wasn't the problem in the first >> place, or the kernel ignores the order. > > Correct. The kernel ignores locking requests because it's a great way > to DOS a machine. For example, mlock() of large blocks of memory is > also not permitted for similar reasons.
Does it ignore such requests in general, or only under certain situations? If the latter, do you know what those situations are? If the former, that seems incredibly bogus. There are plenty of ways to DOS a machine. The main way you prevent DOS by your own authorized users (other than firing them) on linux is by "setrlimit", not by claiming to implement a feature you haven't actually implemented, or by implementing a feature but rendering it completely useless for the purpose it was intended for. RLIMIT_MEMLOCK exists, it has a small default hard limit, and only root can increase that. If root has gone out of its way to grant the postgres user a higher limit, the kernel should respect that, at least up until the situation become truly desperate. However, I don't have any evidence it is being ignored. I just know that locking the shared memory did not improve things, but I didn't verify that shared memory getting swapped out was the problem in the first place. > The way you make sure shared memory doesn't get swapped out is to make > sure it gets used. (i.e. don't give 2GB shared memory when your > database is 100MB). And don't make your shared memory so large that > you're creating significant memory pressure, otherwise the kernel might > choose to swap our your shared memory rather than say the webserver. > > Your shared memory should be reasonably sized, but you should make sure > the kernel has enough "cache" memory it can throw away first. Unfortunately it is hard to know what the kernel considers to be significant memory pressure. My experience (from mostly non-pgsql work) is that kernel has what I would consider enough cache memory to throw away, but for some reason doesn't throw it away but does more counter productive things instead. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
