Re: [PERFORM] Linux machine aggressively clearing cache

2012-09-13 Thread Ross Reedstrom
On Wed, Apr 18, 2012 at 05:09:29PM -0700, Josh Berkus wrote: > On 4/12/12 8:47 AM, Steve Crawford wrote: > > On 03/30/2012 05:51 PM, Josh Berkus wrote: > >> > >> So this turned out to be a Linux kernel issue. Will document it on > >> www.databasesoup.com. > > Anytime soon? About to build two Post

Re: [PERFORM] Linux machine aggressively clearing cache

2012-04-18 Thread Josh Berkus
On 4/12/12 8:47 AM, Steve Crawford wrote: > On 03/30/2012 05:51 PM, Josh Berkus wrote: >> >> So this turned out to be a Linux kernel issue. Will document it on >> www.databasesoup.com. > Anytime soon? About to build two PostgreSQL servers and wondering if you > have uncovered a kernel version or s

Re: [PERFORM] Linux machine aggressively clearing cache

2012-04-12 Thread Steve Crawford
On 03/30/2012 05:51 PM, Josh Berkus wrote: So this turned out to be a Linux kernel issue. Will document it on www.databasesoup.com. Anytime soon? About to build two PostgreSQL servers and wondering if you have uncovered a kernel version or similar issue to avoid. Cheers, Steve -- Sent via

Re: [PERFORM] Linux machine aggressively clearing cache

2012-03-30 Thread Josh Berkus
>> Read cache of course does not need to be flushed and can simply be >> dumped when the memory is needed, and so Linux will keep more or >> less unlimited amounts of read cache until it needs the memory for >> something else > > Right, that's the normal behavior. Except not on this machine

Re: [PERFORM] Linux machine aggressively clearing cache

2012-03-28 Thread Joshua Berkus
> This may just be a typo, but if you really did create write (dirty) > block device cache by writing the pg_dump file somewhere, then that > is what it's supposed to do ;) The pgdump was across the network. So the only caching on the machine was read caching. > Read cache of course does not

Re: [PERFORM] Linux machine aggressively clearing cache

2012-03-27 Thread Dave Crooke
This may just be a typo, but if you really did create write (dirty) block device cache by writing the pg_dump file somewhere, then that is what it's supposed to do ;) Linux is more aggressive about write cache and will allow more of it to build up than e.g. HP-UX which will start to throttle proces

Re: [PERFORM] Linux machine aggressively clearing cache

2012-03-27 Thread Claudio Freire
On Tue, Mar 27, 2012 at 5:06 PM, Joshua Berkus wrote: > In an effort to test this, we deliberately ran a pg_dump.  This did grow the > cache to all available memory, but Linux rapidly cleared the cache (flushing > to disk) down to 25GB within an hour. This would happen if some queries (or some

[PERFORM] Linux machine aggressively clearing cache

2012-03-27 Thread Joshua Berkus
Have run across some memory behavior on Linux I've never seen before. Server running RHEL6 with 96GB of RAM. Kernel 2.6.32 PostgreSQL 9.0 208GB database with fairly random accesses over 50% of the database. Now, here's the weird part: even after a week of uptime, only 21 to 25GB of cache is ev