Re: [PERFORM] Best practice when reindexing in production

2013-05-29 Thread Armand du Plessis
On Wed, May 29, 2013 at 2:26 PM, Magnus Hagander wrote: > On Wed, May 29, 2013 at 8:24 AM, Niels Kristian Schjødt > wrote: > > Hi, > > > > I have a database with quite some data (millions of rows), that is > heavily updated all the time. Once a day I would like to reindex my > database (and maybe

Re: [PERFORM] Problems with pg_locks explosion

2013-04-02 Thread Armand du Plessis
y were idling) Thanks again for all the help and suggestions. Armand On Tue, Apr 2, 2013 at 11:55 AM, Armand du Plessis wrote: > Jumped the gun a bit. the problem still exists like before. But it's > definitely on the right track, below is the output from top in the seconds > be

Re: [PERFORM] Problems with pg_locks explosion

2013-04-02 Thread Armand du Plessis
00 S 5.5 0.0 18:51.81 [migration/5] 48 root 20 0 000 S 5.5 0.0 3:52.93 [kworker/14:0] On Tue, Apr 2, 2013 at 10:16 AM, Armand du Plessis wrote: > Touch wood but I think I found the problem thanks to these pointers. I > checked the vm.zone_reclaim_mo

Re: [PERFORM] Problems with pg_locks explosion

2013-04-02 Thread Armand du Plessis
checking what your sysctl vm.zone_reclaim_mode is set to > - if 1 then override to 0. As Jeff mentioned, this gotcha for larger cpu > number machines has been discussed at length on this list - but still traps > us now and again! > > Cheers > > Mark > > > On 02/04/13 19:3

Re: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
I had my reservations about my almost 0% IO usage on the raid0 array as well. I'm looking at the numbers in atop and it doesn't seem to reflect the aggregate of the volumes as one would expect. I'm just happy I am seeing numbers on the volumes, they're not too bad. One thing I was wondering, as a

Re: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
213187 | 623507632 | 403590060861 | 256853219556 | 185283046675 | 95086454 | 66720609 | 4449894 | 0 | 18 | 12076277760 | 0 | 4689747.806 | 4526.539 | 2013-03-29 16:1 8:09.334432+00 On Tue, Apr 2, 2013 at 8:08 AM, Jeff Janes wrote: > On Monday,

Re: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
Hi Steven, Sounds very familiar. Painfully familiar :( But I really don't know. All I can see is that in this particular configuration the instance has 2 x Intel Xeon E5-2670, eight-core processors. I can't find any info on whether it's flex or round robin. AWS typically don't make the underlying

Re: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
read=0.670" "-> Index Only Scan using profiles_pkey on public.profiles (cost=0.00..1.40 rows=1 width=4) (actual time=0.012..0.015 rows=1 loops=783)" " Output: profiles.id" " Index Cond: (profiles.id = mess

Re: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
r 1, 2013 at 3:35 PM, Armand du Plessis wrote: > >> [Apologies, I first sent this to the incorrect list, postgres-admin, in >> the event you receive it twice] >> >> Hi there, >> >> I'm hoping someone on the list can shed some light on an issue I'm

Fwd: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
Thanks Mark, I had a look at the iostat output (on a 5s interval) and pasted it below. The utilization and waits seems low. Included a sample below, #1 taken during normal operation and then when the locks happen it basically drops to 0 across the board. My (mis)understanding of the IOPS was that

Re: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
3 75 22 0 0 On Tue, Apr 2, 2013 at 1:09 AM, Armand du Plessis wrote: > Thanks for the reply. > > I've now updated the background writer settings to: > > # - Background Writer - > > bgwriter_delay = 200ms # 10-1ms between rounds > bgwriter_lru_m

Re: [PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
y to tune my checkpoint parameters better, if that doesn't work, > show us some vmstat output please > > Vasilis Ventirozos > ------ Forwarded message -- > From: "Armand du Plessis" > Date: Apr 2, 2013 1:37 AM > Subject: [PERFORM] Problems with pg_locks e

[PERFORM] Problems with pg_locks explosion

2013-04-01 Thread Armand du Plessis
[Apologies, I first sent this to the incorrect list, postgres-admin, in the event you receive it twice] Hi there, I'm hoping someone on the list can shed some light on an issue I'm having with our Postgresql cluster. I'm literally tearing out my hair and don't have a deep enough understanding of