Re: [PERFORM] Volatility - docs vs behaviour?

2014-06-30 Thread Tom Lane
Craig Ringer writes: > I was unaware that the planner made any attempt to catch users' errors > in marking the strictness of functions. I thought it pretty much trusted > the user not to lie about the mutability of functions invoked > indirectly. I'm not really sure where in the inlining code to l

Re: [PERFORM] Volatility - docs vs behaviour?

2014-06-30 Thread Craig Ringer
On 06/30/2014 11:49 PM, Tom Lane wrote: > Craig Ringer writes: >> The docs say: > >> "For best optimization results, you should label your functions with the >> strictest volatility category that is valid for them." > > Yeah ... > >> ... but I recall discussion here suggesting that in fact IMMU

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 4:04 PM, Tom Lane wrote: > Ah ... that's more like a number I can believe something would have > trouble coping with. Did you see a noticeable slowdown with this? > Now that we've seen that number, of course it's possible there was an > even higher peak occurring when you sa

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Andres Freund
On 2014-06-30 19:04:20 -0400, Tom Lane wrote: > Jeff Frost writes: > >>> So it seems like we have a candidate explanation. I'm a bit surprised > >>> that StandbyReleaseLocks would get this slow if there are only a dozen > >>> AccessExclusiveLocks in place at any one time, though. Perhaps that >

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Tom Lane
Jeff Frost writes: >>> So it seems like we have a candidate explanation. I'm a bit surprised >>> that StandbyReleaseLocks would get this slow if there are only a dozen >>> AccessExclusiveLocks in place at any one time, though. Perhaps that >>> was a low point and there are often many more? > Si

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 1:46 PM, Jeff Frost wrote: >> So it seems like we have a candidate explanation. I'm a bit surprised >> that StandbyReleaseLocks would get this slow if there are only a dozen >> AccessExclusiveLocks in place at any one time, though. Perhaps that >> was a low point and there

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 1:39 PM, Tom Lane wrote: > > >> Another item of note is the system catalogs are quite bloated: >> Would that cause the replica to spin on StandbyReleaseLocks? > > AFAIK, no. It's an unsurprising consequence of heavy use of short-lived > temp tables though. > Yah, this h

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Tom Lane
Jeff Frost writes: > On Jun 30, 2014, at 1:15 PM, Andres Freund wrote: >> So these are probably relations created in uncommitted >> transactions. Possibly ON COMMIT DROP temp tables? > That would make sense. There are definitely quite a few of those being used. Uh-huh. I doubt that the mechan

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 1:15 PM, Andres Freund wrote: > On 2014-06-30 12:57:56 -0700, Jeff Frost wrote: >> >> On Jun 30, 2014, at 12:54 PM, Matheus de Oliveira >> wrote: >> >>> >>> On Mon, Jun 30, 2014 at 4:42 PM, Jeff Frost wrote: >>> And if you go fishing in pg_class for any of the oids, you

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Andres Freund
On 2014-06-30 12:57:56 -0700, Jeff Frost wrote: > > On Jun 30, 2014, at 12:54 PM, Matheus de Oliveira > wrote: > > > > > On Mon, Jun 30, 2014 at 4:42 PM, Jeff Frost wrote: > > And if you go fishing in pg_class for any of the oids, you don't find > > anything: > > > > That is probably becaus

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 12:54 PM, Matheus de Oliveira wrote: > > On Mon, Jun 30, 2014 at 4:42 PM, Jeff Frost wrote: > And if you go fishing in pg_class for any of the oids, you don't find > anything: > > That is probably because you are connected in the wrong database. Once you > connect to th

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Matheus de Oliveira
On Mon, Jun 30, 2014 at 4:42 PM, Jeff Frost wrote: > And if you go fishing in pg_class for any of the oids, you don't find > anything: That is probably because you are connected in the wrong database. Once you connect to the database of interest, you don't even need to query pg_class, just cast

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 12:32 PM, Tom Lane wrote: > Jeff Frost writes: >> Sampling pg_locks on the primary shows ~50 locks with ExclusiveLock mode: > >> mode | count >> --+--- >> AccessExclusiveLock |11 >> AccessShareLock | 2089 >>

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Tom Lane
Jeff Frost writes: > Sampling pg_locks on the primary shows ~50 locks with ExclusiveLock mode: >mode | count > --+--- > AccessExclusiveLock |11 > AccessShareLock | 2089 > ExclusiveLock|46 > RowExclusiveLock

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 12:17 PM, Jeff Frost wrote: >> >> already is quite helpful. >> >> What are you doing on that system? Is there anything requiring large >> amounts of access exclusive locks on the primary? Possibly large amounts >> of temporary relations? > > > The last time we did a 100% l

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 11:39 AM, Andres Freund wrote: > On 2014-06-30 11:34:52 -0700, Jeff Frost wrote: >> On Jun 30, 2014, at 10:29 AM, Soni M wrote: > >>> It is >>> 96.62% postgres [.] StandbyReleaseLocks >>> as Jeff said. It runs quite long time, more than 5 minutes i think >>> >

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Pavel Stehule
2014-06-30 20:34 GMT+02:00 Jeff Frost : > On Jun 30, 2014, at 10:29 AM, Soni M wrote: > > > > > On Tue, Jul 1, 2014 at 12:14 AM, Andres Freund > wrote: > >> >> My guess it's a spinlock, probably xlogctl->info_lck via >> RecoveryInProgress(). Unfortunately inline assembler doesn't always seem >>

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Andres Freund
On 2014-06-30 11:34:52 -0700, Jeff Frost wrote: > On Jun 30, 2014, at 10:29 AM, Soni M wrote: > > It is > > 96.62% postgres [.] StandbyReleaseLocks > > as Jeff said. It runs quite long time, more than 5 minutes i think > > > > i also use hot standby. we have 4 streaming replica,

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 10:29 AM, Soni M wrote: > > > > On Tue, Jul 1, 2014 at 12:14 AM, Andres Freund wrote: > > My guess it's a spinlock, probably xlogctl->info_lck via > RecoveryInProgress(). Unfortunately inline assembler doesn't always seem > to show up correctly in profiles... > > What wo

Re: [PERFORM] Guidelines on best indexing strategy for varying searches on 20+ columns

2014-06-30 Thread Jeff Janes
On Wed, Jun 25, 2014 at 1:48 AM, Niels Kristian Schjødt wrote: > Hi, > I’m running a search engine for cars. It’s backed by a postgresql 9.3 > installation. > > Now I’m unsure about the best approach/strategy on doing index optimization > for the fronted search. > > The problem: > > The table co

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Soni M
On Tue, Jul 1, 2014 at 12:14 AM, Andres Freund wrote: > > My guess it's a spinlock, probably xlogctl->info_lck via > RecoveryInProgress(). Unfortunately inline assembler doesn't always seem > to show up correctly in profiles... > > What worked for me was to build with -fno-omit-frame-pointer - th

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Andres Freund
On 2014-06-30 19:14:24 +0300, Heikki Linnakangas wrote: > On 06/30/2014 05:46 PM, Soni M wrote: > >Here's what 'perf top' said on streaming replica : > > > >Samples: 26K of event 'cpu-clock', Event count (approx.): 19781 > > 95.97% postgres [.] 0x002210f3 > >

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Jeff Frost
On Jun 30, 2014, at 9:14 AM, Heikki Linnakangas wrote: > On 06/30/2014 05:46 PM, Soni M wrote: >> Here's what 'perf top' said on streaming replica : >> >> Samples: 26K of event 'cpu-clock', Event count (approx.): 19781 >> 95.97% postgres [.] 0x002210f3 > >

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Heikki Linnakangas
On 06/30/2014 05:46 PM, Soni M wrote: Here's what 'perf top' said on streaming replica : Samples: 26K of event 'cpu-clock', Event count (approx.): 19781 95.97% postgres [.] 0x002210f3 Ok, so it's stuck doing something.. Can you get build with debug symbol

Re: [PERFORM] Volatility - docs vs behaviour?

2014-06-30 Thread Tom Lane
Craig Ringer writes: > The docs say: > "For best optimization results, you should label your functions with the > strictest volatility category that is valid for them." Yeah ... > ... but I recall discussion here suggesting that in fact IMMUTABLE > functions may not be inlined where you'd expec

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Soni M
Here's what 'perf top' said on streaming replica : Samples: 26K of event 'cpu-clock', Event count (approx.): 19781 95.97% postgres [.] 0x002210f3 0.41% perf [.] 0x0005f225 0.39% libc-2.12.so

Re: [PERFORM] GIST optimization to limit calls to operator on sub nodes

2014-06-30 Thread Tom Lane
Pujol Mathieu writes: > Le 29/06/2014 22:30, Tom Lane a écrit : >> I don't actually understand what's being requested here that the >> NotConsistent case doesn't already cover. > The NotConsistent case is correctly covered, the sub nodes are not > tested because I know that no child could pass t

[PERFORM] Volatility - docs vs behaviour?

2014-06-30 Thread Craig Ringer
Hi all The docs say: "For best optimization results, you should label your functions with the strictest volatility category that is valid for them." http://www.postgresql.org/docs/current/interactive/xfunc-volatility.html ... but I recall discussion here suggesting that in fact IMMUTABLE functi

Re: [PERFORM] GIST optimization to limit calls to operator on sub nodes

2014-06-30 Thread Pujol Mathieu
Le 29/06/2014 22:30, Tom Lane a écrit : Emre Hasegeli writes: Pujol Mathieu : I made my own index to handle specific data and operators. It works pretty fine but I wonder if it was possible to optimize it. When I run my operator on a GIST node (in the method gist_range_consistent) it returns

Re: [PERFORM] Guidelines on best indexing strategy for varying searches on 20+ columns

2014-06-30 Thread Niels Kristian Schjødt
Thanks for your suggestions, very useful. See comments inline: Den 25/06/2014 kl. 23.48 skrev Merlin Moncure : > On Wed, Jun 25, 2014 at 3:48 AM, Niels Kristian Schjødt > wrote: >> Hi, >> I’m running a search engine for cars. It’s backed by a postgresql 9.3 >> installation. >> >> Now I’m unsur

Re: [PERFORM] GIST optimization to limit calls to operator on sub nodes

2014-06-30 Thread Pujol Mathieu
Le 29/06/2014 22:14, Emre Hasegeli a écrit : Pujol Mathieu : Hello, I already post my question in the General Mailing list, but without succeed so I try this one that seems to me more specialized. My question is about GIST index. I made my own index to handle specific data and operators. It wor

Re: [PERFORM] Postgres Replaying WAL slowly

2014-06-30 Thread Heikki Linnakangas
On 06/29/2014 03:43 PM, Soni M wrote: top and sar says 100% cpu usage of one core, no sign of I/O wait. Hmm, I wonder what it's doing then... If you have "perf" installed on the system, you can do "perf top" to get a quick overlook of where the CPU time is spent. - Heikki -- Sent via pgs