On Tue, Aug 30, 2011 at 9:05 PM, Scott Marlowe wrote:
> Well, we had zone reclaim mode autoset to 1, and we had to turn it off
> to get decent performance with postgresql. Machine was a quad
> dodecacore Magny Cours, so 48 cores with 128G RAM. RAID controller is
> an Areca 1680 with BBU, 34 15kR
On Tue, Aug 30, 2011 at 10:05 PM, Scott Marlowe wrote:
> On Tue, Aug 30, 2011 at 8:36 PM, mark wrote:
>> To the broader list, regarding troubles with kswap. I am curious to what
>> others seeing from /proc/zoneinfo for DMA pages (not dma32 or normal) -
>> basically if it sits at 1 or not. Settin
On Tue, Aug 30, 2011 at 8:36 PM, mark wrote:
> To the broader list, regarding troubles with kswap. I am curious to what
> others seeing from /proc/zoneinfo for DMA pages (not dma32 or normal) -
> basically if it sits at 1 or not. Setting swappiness to 0 did not have any
> affect for us on kswap i
On Tue, Aug 30, 2011 at 8:36 PM, mark wrote:
>
> Scott,
> 1000 max connections ? I thought that was several times more than
> recommended these days, even for 24 or 48 core machines. Or am I living in
> the past ? (I admit that my most recent runs of pgbench showed that best
> throughput at around
> -Original Message-
> From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-
> ow...@postgresql.org] On Behalf Of Scott Marlowe
> Sent: Tuesday, August 30, 2011 3:52 AM
> To: Sim Zacks
> Cc: pgsql-general@postgresql.org
> Subject: Re: [GENERAL] heavy s
On Tue, Aug 30, 2011 at 5:05 PM, Lonni J Friedman wrote:
> On Tue, Aug 30, 2011 at 2:55 PM, John R Pierce wrote:
>> On 08/30/11 12:18 PM, Tom Lane wrote:
>
> total used free shared buffers cached
> >> Mem: 56481 55486 995
On Tue, Aug 30, 2011 at 2:55 PM, John R Pierce wrote:
> On 08/30/11 12:18 PM, Tom Lane wrote:
total used free shared buffers cached
>> Mem: 56481 55486 995 0 15
>> 53298
>> -/+ buffers/cache: 2
On 08/30/11 12:18 PM, Tom Lane wrote:
total used free sharedbuffers cached
>> Mem: 56481 55486995 0 15 53298
>> -/+ buffers/cache: 2172 54309
>> Swap: 1099 18 1081
> This is totall
Merlin Moncure writes:
> On Tue, Aug 30, 2011 at 11:17 AM, Lonni J Friedman wrote:
>> In the past 18 hours, swap usage has nearly doubled on systemA:
>> $ free -m
>> total used free sharedbuffers cached
>> Mem: 56481 56210271 0
On Tue, Aug 30, 2011 at 11:17 AM, Lonni J Friedman wrote:
> On Mon, Aug 29, 2011 at 5:42 PM, Tom Lane wrote:
>> Lonni J Friedman writes:
>>> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>>> I'm experiencing significant swap usage growth over time. All of them
>>> have fa
On Mon, Aug 29, 2011 at 5:42 PM, Tom Lane wrote:
> Lonni J Friedman writes:
>> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>> I'm experiencing significant swap usage growth over time. All of them
>> have fairly substantial amounts of RAM (not including swap), yet the
>>
On Tue, Aug 30, 2011 at 3:00 AM, Boszormenyi Zoltan wrote:
> Hi,
>
> 2011-08-29 22:36 keltezéssel, Lonni J Friedman írta:
>> ... I read that
>> (max_connections * work_mem) should never exceed physical RAM, and if
>> that's accurate, then I suspect that's the root of my problem on
>> systemA (bel
On Mon, Aug 29, 2011 at 6:54 PM, peixubin wrote:
> You should monitor PageTables value in /proc/meminfo.if the value larger than
> 1G,I Suggest enable hugepages .
>
> To monitor PageTables:
> # cat /proc/meminfo |grep -i pagetables
$ cat /proc/meminfo |grep -i pagetables
PageTables: 608
On Tue, Aug 30, 2011 at 1:26 AM, Greg Smith wrote:
> I doubt this has anything to do with your problem, just pointing this out as
> future guidance. Until there's a breakthrough in the PostgreSQL buffer
> cache code, there really is no reason to give more than 8GB of dedicated
> memory to the dat
Lonni J Friedman wrote:
> ok, I'll do my best to capture this data, and then reply back.
If using linux, you should find interesting data on per-process swap and
memory usage in /proc/${pid}/smaps
Also consider the script here:
http://northernmost.org/blog/find-out-what-is-using-your-swa
It is recommended to identify the processes using up high work_mem and try
to set work_mem to higher value at the session level.
I this case, all the connections using up maximum work_mem is the potential
threat. As said by Zoltan, work_mem is very high and shared_buffers as well.
Other considera
Hi,
2011-08-29 22:36 keltezéssel, Lonni J Friedman írta:
> ... I read that
> (max_connections * work_mem) should never exceed physical RAM, and if
> that's accurate, then I suspect that's the root of my problem on
> systemA (below).
work_mem is process-local memory so
(max_connections * work_me
On Tue, Aug 30, 2011 at 2:50 AM, Sim Zacks wrote:
>
> Instead of restarting the database try swapoff -a && swapon -a and see if
> that helps performance. If it is that little swap in use, it might be
> something else clogging up the works.
Check to see if kswapd is going crazy or not. If it is,
On Tue, Aug 30, 2011 at 2:50 AM, Sim Zacks wrote:
>
> On a machine with lots of memory, I've run into pathological behaviour
> with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
> eating up CPU and swap io like mad, while doing essentially nothing.
> Setting swappiness to 0 dela
Yes, a few hundred MB of swap, and its definitely making a huge
difference. Upon restarting postgres, its all freed up, and then perf
is good again. Also, this box only has 1GB of swap total, so its
never going to get up a few dozen GB.
Anyway, here's some of top output f
On a machine with lots of memory, I've run into pathological behaviour
with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
eating up CPU and swap io like mad, while doing essentially nothing.
Setting swappiness to 0 delayed this behaviour but did not stop i
On Tue, Aug 30, 2011 at 10:39 AM, Scott Marlowe wrote:
> On Tue, Aug 30, 2011 at 12:54 AM, Florian Weimer wrote:
>> * Scott Marlowe:
>>
>>> On a machine with lots of memory, I've run into pathological behaviour
>>> with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
>>> eating u
On Tue, Aug 30, 2011 at 12:54 AM, Florian Weimer wrote:
> * Scott Marlowe:
>
>> On a machine with lots of memory, I've run into pathological behaviour
>> with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
>> eating up CPU and swap io like mad, while doing essentially nothing.
>>
* Scott Marlowe:
> On a machine with lots of memory, I've run into pathological behaviour
> with both the RHEL 5 and Ubuntu 10.04 kernels where the kswapd starts
> eating up CPU and swap io like mad, while doing essentially nothing.
> Setting swappiness to 0 delayed this behaviour but did not stop
On 08/29/2011 06:12 PM, Lonni J Friedman wrote:
OK, I'll reduce it to 10GB and see if there's any noticable change in
performance. thanks
I've never heard a report of a Linux system using more than 8GB of
shared_buffers usefully, and peak performance on systems I've tested has
sometimes
On Mon, Aug 29, 2011 at 5:01 PM, Alan Hodgson wrote:
> On August 29, 2011 02:34:26 PM you wrote:
>> On Mon, Aug 29, 2011 at 3:46 PM, Alan Hodgson wrote:
>> > On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
>> >> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>> >> I'm
You should monitor PageTables value in /proc/meminfo.if the value larger than
1G,I Suggest enable hugepages .
To monitor PageTables:
# cat /proc/meminfo |grep -i pagetables
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www
On Mon, Aug 29, 2011 at 5:42 PM, Tom Lane wrote:
> Lonni J Friedman writes:
>> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>> I'm experiencing significant swap usage growth over time. All of them
>> have fairly substantial amounts of RAM (not including swap), yet the
>>
Lonni J Friedman writes:
> I have several Linux-x68_64 based dedicated PostgreSQL servers where
> I'm experiencing significant swap usage growth over time. All of them
> have fairly substantial amounts of RAM (not including swap), yet the
> amount of swap that postgres is using ramps up over time
On August 29, 2011 02:34:26 PM you wrote:
> On Mon, Aug 29, 2011 at 3:46 PM, Alan Hodgson wrote:
> > On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
> >> I have several Linux-x68_64 based dedicated PostgreSQL servers where
> >> I'm experiencing significant swap usage growth over time.
> >
On Mon, Aug 29, 2011 at 3:17 PM, Merlin Moncure wrote:
> On Mon, Aug 29, 2011 at 4:45 PM, Lonni J Friedman wrote:
>>> using any C code in the backend? this includes 3rd party libraries
>>> which link in C, including postgis, pljava, xml2, etc. Any features
>>> being used not included in the stan
On Mon, Aug 29, 2011 at 4:45 PM, Lonni J Friedman wrote:
>> using any C code in the backend? this includes 3rd party libraries
>> which link in C, including postgis, pljava, xml2, etc. Any features
>> being used not included in the standard core distribution are
>> interesting.
>
> Nope, nothing
On Mon, Aug 29, 2011 at 2:51 PM, Scott Marlowe wrote:
> On Mon, Aug 29, 2011 at 3:38 PM, Lonni J Friedman wrote:
>> On Mon, Aug 29, 2011 at 2:24 PM, Scott Marlowe
>> wrote:
>>> On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman
>>> wrote:
On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson wro
On Mon, Aug 29, 2011 at 3:38 PM, Lonni J Friedman wrote:
> On Mon, Aug 29, 2011 at 2:24 PM, Scott Marlowe
> wrote:
>> On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman wrote:
>>> On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson wrote:
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
>>
On Mon, Aug 29, 2011 at 2:38 PM, Merlin Moncure wrote:
> On Mon, Aug 29, 2011 at 3:36 PM, Lonni J Friedman wrote:
>> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>> I'm experiencing significant swap usage growth over time. All of them
>> have fairly substantial amounts of
On Mon, Aug 29, 2011 at 2:24 PM, Scott Marlowe wrote:
> On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman wrote:
>> On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson wrote:
>>> On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
I have several Linux-x68_64 based dedicated PostgreSQL servers
On Mon, Aug 29, 2011 at 3:36 PM, Lonni J Friedman wrote:
> I have several Linux-x68_64 based dedicated PostgreSQL servers where
> I'm experiencing significant swap usage growth over time. All of them
> have fairly substantial amounts of RAM (not including swap), yet the
> amount of swap that post
On Mon, Aug 29, 2011 at 3:46 PM, Alan Hodgson wrote:
> On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
>> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>> I'm experiencing significant swap usage growth over time.
>
> It's the Linux kernel that does it, not PostgreSQL.
On Mon, Aug 29, 2011 at 2:57 PM, Lonni J Friedman wrote:
> On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson wrote:
>> On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
>>> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>>> I'm experiencing significant swap usage growth ove
On Mon, Aug 29, 2011 at 1:46 PM, Alan Hodgson wrote:
> On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
>> I have several Linux-x68_64 based dedicated PostgreSQL servers where
>> I'm experiencing significant swap usage growth over time.
>
> It's the Linux kernel that does it, not PostgreSQL.
On August 29, 2011 01:36:07 PM Lonni J Friedman wrote:
> I have several Linux-x68_64 based dedicated PostgreSQL servers where
> I'm experiencing significant swap usage growth over time.
It's the Linux kernel that does it, not PostgreSQL. Set vm.swappiness=0
(usually in /etc/sysctl.conf) and put t
I have several Linux-x68_64 based dedicated PostgreSQL servers where
I'm experiencing significant swap usage growth over time. All of them
have fairly substantial amounts of RAM (not including swap), yet the
amount of swap that postgres is using ramps up over time and
eventually hurts performance
42 matches
Mail list logo
