Re: [HACKERS] format() with embedded to_char() formatter

2010-11-24 Thread Itagaki Takahiro
On Tue, Nov 23, 2010 at 03:08, Pavel Stehule wrote: >>> Or is the intention to replicate the parser's >>> overloaded-function-resolution behavior at runtime?  That seems awkward, >>> duplicative, slow, and probably prone to security issues (think >>> search_path). >> >> Ick. I've thought lookup_a

Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-24 Thread Maciek Sakrejda
I've run your test and I can confirm the error. I've looked at the protocol traffic with Wireshark, and the back-end is clearly lying about the format of the results in this particular case: as you stated, the row description says text, but the data is in binary. I also wrote a simple Java program

[HACKERS] How strings are sorted by LC_COLLATE specifically?

2010-11-24 Thread Chang Chao
Hello Dear All. How strings are sorted when LC_COLLATE = ja_JP.UTF-8. I tried to read the documention on that,but there are just a few words, like LC_COLLATE determines string sort order, Is there a specific reference about this? So I can implement an equivalent string sort function in JAVA. becau

[HACKERS] pg_execute_from_file review

2010-11-24 Thread Joshua Tolley
I've just looked at pg_execute_from_file[1]. The idea here is to execute all the SQL commands in a given file. My comments: * It applies well enough, and builds fine * It seems to work, and I've not come up with a way to make it break * It seems useful, and to follow the limited design discussion

Re: [HACKERS] contrib: auth_delay module

2010-11-24 Thread KaiGai Kohei
(2010/11/19 16:57), KaiGai Kohei wrote: (2010/11/18 2:17), Robert Haas wrote: On Wed, Nov 17, 2010 at 10:32 AM, Ross J. Reedstrom wrote: On Tue, Nov 16, 2010 at 09:41:37PM -0500, Robert Haas wrote: On Tue, Nov 16, 2010 at 8:15 PM, KaiGai Kohei wrote: If we don't need a PoC module for each new

Re: [HACKERS] Extensions, this time with a patch

2010-11-24 Thread Itagaki Takahiro
On Thu, Nov 25, 2010 at 05:02, Dimitri Fontaine wrote: > Something like the attached, version 5 of the patch? I've been using the > function name ParseConfigFp because the internal parameter was called fp > in the previous function body. I suppose that could easily be changed at > commit time if n

Re: [HACKERS] Label switcher function

2010-11-24 Thread KaiGai Kohei
The attached patch is a revised one. It provides two hooks; the one informs core PG whether the supplied function needs to be hooked, or not. the other is an actual hook on prepare, start, end and abort of function invocations. typedef bool (*needs_function_call_type)(Oid fn_oid); typedef vo

Re: [HACKERS] security hooks on object creation

2010-11-24 Thread KaiGai Kohei
The attached patch is a revised patch. - The utils/hooks.h was renamed to catalog/objectaccess.h - Numeric in the tail of InvokeObjectAccessHook0() has gone. - Fixed bug in ATExecAddColumn; it gave AttributeRelationId to the hook instead of RelationRelationId. In addition, I found that we didn'

Re: [HACKERS] Tab completion for view triggers in psql

2010-11-24 Thread Josh Kupershmidt
On Tue, Nov 23, 2010 at 10:21 PM, David Fetter wrote: > Please find attached a patch changing both this and "updateable" to > "updatable," also per the very handy git grep I just learned about :) I looked a little more at this patch today. I didn't find any serious problems, though it would have

Re: [HACKERS] Per-column collation, work in progress

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 3:37 PM, Peter Eisentraut wrote: > On tor, 2010-10-14 at 22:54 -0400, Robert Haas wrote: >> It seems you've falsified the header comment in >> pathkeys_useful_for_merging(), although I guess it's already false >> because it doesn't seem to have been updated for the NULLS AS

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Daniel Farina
On Wed, Nov 24, 2010 at 7:21 PM, Bruce Momjian wrote: > What are we adding a pl/pgsql dependency for?  What is the benefit that > will warrant requiring people who disable plpgsql to enable it for > restores? There are two use cases I want to cover: 1) It should be possible to restore a dump mad

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Bruce Momjian
bruce wrote: > Alvaro Herrera wrote: > > Excerpts from Bruce Momjian's message of nov 24 19:04:30 -0300 2010: > > > Bruce Momjian wrote: > > > > > > OK, we will not call inet_ntop() at all. I moved the CIDR part of > > > > adt/inet_net_ntop.c into adt/inet_cidr_ntop.c, and moved the remaining > >

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Bruce Momjian
Daniel Farina wrote: > On Wed, Nov 24, 2010 at 7:03 PM, Bruce Momjian wrote: > > Daniel Farina wrote: > >> On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian wrote: > >> > Robert Haas wrote: > >> >> With respect to the syntax itself, I have mixed feelings. ?On the one > >> >> hand, I'm a big fan of C

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Daniel Farina
On Wed, Nov 24, 2010 at 7:03 PM, Bruce Momjian wrote: > Daniel Farina wrote: >> On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian wrote: >> > Robert Haas wrote: >> >> With respect to the syntax itself, I have mixed feelings. ?On the one >> >> hand, I'm a big fan of CREATE IF NOT EXISTS and DROP IF E

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Bruce Momjian
Daniel Farina wrote: > On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian wrote: > > Robert Haas wrote: > >> With respect to the syntax itself, I have mixed feelings. ?On the one > >> hand, I'm a big fan of CREATE IF NOT EXISTS and DROP IF EXISTS > >> precisely because I believe they handle many commo

[HACKERS] Re: [COMMITTERS] pgsql: Document that a CHECKPOINT before taking a file system snapshot

2010-11-24 Thread Bruce Momjian
C?dric Villemain wrote: > 2010/11/25 Bruce Momjian : > > Document that a CHECKPOINT before taking a file system snapshot can > > reduce recovery time. > > I didn't follow that on -hackers, but : > > * checkpoint take place in the pg_start_backup process (before it > releases the hand, so before y

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Bruce Momjian
Alvaro Herrera wrote: > Excerpts from Bruce Momjian's message of nov 24 19:04:30 -0300 2010: > > Bruce Momjian wrote: > > > > OK, we will not call inet_ntop() at all. I moved the CIDR part of > > > adt/inet_net_ntop.c into adt/inet_cidr_ntop.c, and moved the remaining > > > "net" part to /port/in

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Robert Haas
On Nov 24, 2010, at 5:49 PM, Tom Lane wrote: > Jeff Janes writes: >> I've played a bit with hash_search_with_hash_value and found that most >> of the time is spent on shared hash tables, not private ones. And the >> time attributed to it for the shared hash tables mostly seems to be >> due to th

[HACKERS] Re: [COMMITTERS] pgsql: Document that a CHECKPOINT before taking a file system snapshot

2010-11-24 Thread Cédric Villemain
2010/11/25 Bruce Momjian : > Document that a CHECKPOINT before taking a file system snapshot can > reduce recovery time. I didn't follow that on -hackers, but : * checkpoint take place in the pg_start_backup process (before it releases the hand, so before you can start snapshoting) * we used to

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Daniel Farina
On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian wrote: > Robert Haas wrote: >> With respect to the syntax itself, I have mixed feelings.  On the one >> hand, I'm a big fan of CREATE IF NOT EXISTS and DROP IF EXISTS >> precisely because I believe they handle many common cases that people >> want in

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mié nov 24 19:04:30 -0300 2010: > Bruce Momjian wrote: > > OK, we will not call inet_ntop() at all. I moved the CIDR part of > > adt/inet_net_ntop.c into adt/inet_cidr_ntop.c, and moved the remaining > > "net" part to /port/inet_net_ntop.c. > Applied. Th

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Bruce Momjian
Robert Haas wrote: > With respect to the syntax itself, I have mixed feelings. On the one > hand, I'm a big fan of CREATE IF NOT EXISTS and DROP IF EXISTS > precisely because I believe they handle many common cases that people > want in real life without much hullabaloo. But, there's clearly some

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Michael Glaesemann
On Nov 24, 2010, at 15:28 , Marti Raudsepp wrote: > On Wed, Nov 24, 2010 at 21:52, Tom Lane wrote: >> Bruce Momjian writes: >>> Notice the to_date()'s were not converted to constants in EXPLAIN so >>> they are evaluated for every row. to_date() is marked STABLE. > >> No. This is per expectat

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Jeff Janes
On Wed, Nov 24, 2010 at 2:34 PM, Robert Haas wrote: > On Wed, Nov 24, 2010 at 5:33 PM, Jeff Janes wrote: >> >> I've played a bit with hash_search_with_hash_value and found that most >> of the time is spent on shared hash tables, not private ones.  And the >> time attributed to it for the shared h

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Tom Lane
Magnus Hagander writes: > On Wed, Nov 24, 2010 at 23:45, Tom Lane wrote: >> Alvaro Herrera writes: >>> This was stuck in the moderation queue because of message size limit (30 >>> kB).  Is it worth increasing that value? >> >> Evidently we should.  pgindent and copyright-update commits are like

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Tom Lane
Jeff Janes writes: > I've played a bit with hash_search_with_hash_value and found that most > of the time is spent on shared hash tables, not private ones. And the > time attributed to it for the shared hash tables mostly seems to be > due to the time it takes to fight cache lines away from other

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Bruce Momjian
Magnus Hagander wrote: > On Wed, Nov 24, 2010 at 23:45, Tom Lane wrote: > > Alvaro Herrera writes: > >> Excerpts from Peter Eisentraut's message of mar nov 23 17:52:18 -0300 2010: > >>> Remove useless whitespace at end of lines > > > >> This was stuck in the moderation queue because of message si

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Magnus Hagander
On Wed, Nov 24, 2010 at 23:45, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Peter Eisentraut's message of mar nov 23 17:52:18 -0300 2010: >>> Remove useless whitespace at end of lines > >> This was stuck in the moderation queue because of message size limit (30 >> kB).  Is it worth i

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Peter Eisentraut's message of mar nov 23 17:52:18 -0300 2010: >> Remove useless whitespace at end of lines > This was stuck in the moderation queue because of message size limit (30 > kB). Is it worth increasing that value? Evidently we should. pgindent a

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 5:42 PM, Tom Lane wrote: > Robert Haas writes: >> I don't see anything for BUS OUTSTANDING.  For CACHE and MISS I have >> some options: > >> DATA_CACHE_MISSES: (counter: all) >> L3_CACHE_MISSES: (counter: all) > > Those two look promising, though I can't claim to be an exp

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > I don't see anything for BUS OUTSTANDING. For CACHE and MISS I have > some options: > DATA_CACHE_MISSES: (counter: all) > L3_CACHE_MISSES: (counter: all) Those two look promising, though I can't claim to be an expert. regards, tom lane -- Sent vi

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 23:34:46 Robert Haas wrote: > On Wed, Nov 24, 2010 at 5:33 PM, Jeff Janes wrote: > > On Wed, Nov 24, 2010 at 1:19 PM, Andres Freund wrote: > >> On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: > >>> On Wednesday 24 November 2010 21:24:43 Robert Haas wrote:

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 5:33 PM, Jeff Janes wrote: > On Wed, Nov 24, 2010 at 1:19 PM, Andres Freund wrote: >> On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: >>> On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: >>> >>> Recarding LWLockAcquire costs: >>> Yes, its pretty notice

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Jeff Janes
On Wed, Nov 24, 2010 at 1:19 PM, Andres Freund wrote: > On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: >> On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: >> >> Recarding LWLockAcquire costs: >> Yes, its pretty noticeable - on loads of different usages. On a bunch of >> produ

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 5:15 PM, Andres Freund wrote: > On Wednesday 24 November 2010 23:03:48 Tom Lane wrote: >> Robert Haas writes: >> > On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane wrote: >> >> (You might be able to confirm or disprove this theory if you ask >> >> oprofile to count memory access

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 23:03:48 Tom Lane wrote: > Robert Haas writes: > > On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane wrote: > >> (You might be able to confirm or disprove this theory if you ask > >> oprofile to count memory access stalls instead of CPU clock cycles...) > > > > I don't see

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Bruce Momjian
Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian writes: > > > I assume you are suggesting to use our inet_net_ntop() even if the > > > system has inet_ntop(). > > > > If you're going to have code to do the former, it doesn't seem to be > > worth the trouble to also have code that does th

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane wrote: >> (You might be able to confirm or disprove this theory if you ask >> oprofile to count memory access stalls instead of CPU clock cycles...) > I don't see an event for that. You probably want something involving cache misse

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane wrote: > (You might be able to confirm or disprove this theory if you ask > oprofile to count memory access stalls instead of CPU clock cycles...) I don't see an event for that. # opcontrol --list-events | grep STALL INSTRUCTION_FETCH_STALL: (counter: al

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 22:25:45 Tom Lane wrote: > Robert Haas writes: > > On Nov 24, 2010, at 4:05 PM, Andres Freund wrote: > >> Yes, but only once. Also scrubbing a page is faster than copying it... > >> (and there were patches floating around to do that in advance, not sure > >> if they

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 22:18:08 Robert Haas wrote: > On Nov 24, 2010, at 4:05 PM, Andres Freund wrote: > >>> Won't this just cause loads of additional pagefaults after fork() when > >>> those pages are used the first time and then a second time when first > >>> written to (to copy it)? > >>

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > On Nov 24, 2010, at 4:05 PM, Andres Freund wrote: >> Yes, but only once. Also scrubbing a page is faster than copying it... (and >> there were patches floating around to do that in advance, not sure if they >> got >> integrated into mainline linux) > I'm not following -

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: > On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: > > I'd like to get access to a box with (a lot) more cores, to see > > whether the lock stuff moves up in the profile. A big chunk of that > > hash_search_with_hash_value overhe

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Nov 24, 2010, at 4:05 PM, Andres Freund wrote: >>> >>> Won't this just cause loads of additional pagefaults after fork() when >>> those pages are used the first time and then a second time when first >>> written to (to copy it)? >> >> Aren't we incurring those page faults anyway, for whatever

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: > I'd like to get access to a box with (a lot) more cores, to see > whether the lock stuff moves up in the profile. A big chunk of that > hash_search_with_hash_value overhead is coming from > LockAcquireExtended. The __strcmp_sse2 is almos

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Tom Lane
Robert Haas writes: > I did some profiling of pgbench -j 36 -c 36 -T 500 banging on my > two-core desktop box - with synchronous_commit turned off to keep the > fsyncs from dominating the profile - and got these results: > 29634 4.7124 postgres base_yyparse Seems like pgbenc

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 21:54:53 Robert Haas wrote: > On Wed, Nov 24, 2010 at 3:53 PM, Andres Freund wrote: > > On Wednesday 24 November 2010 21:47:32 Robert Haas wrote: > >> On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane wrote: > >> > Robert Haas writes: > >> >> Full results, and call graph, a

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 24, 2010 at 3:53 PM, Andres Freund wrote: >>> The idea I had was to go the other way and say, hey, if these hash >>> tables can't be expanded anyway, let's put them on the BSS instead of >>> heap-allocating them. >> Won't this just cause loads of additional page

[HACKERS] Regex code versus Unicode chars beyond codepoint 255

2010-11-24 Thread Tom Lane
Bug #5766 points out that we're still not there yet in terms of having sane behavior for locale-specific regex operations in Unicode encoding. The reason it's not working is that regc_locale does this to expand the set of characters that are considered to match [[:alnum:]] : /* *

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 3:53 PM, Andres Freund wrote: > On Wednesday 24 November 2010 21:47:32 Robert Haas wrote: >> On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane wrote: >> > Robert Haas writes: >> >> Full results, and call graph, attached.  The first obvious fact is >> >> that most of the memset ov

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 21:47:32 Robert Haas wrote: > On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane wrote: > > Robert Haas writes: > >> Full results, and call graph, attached. The first obvious fact is > >> that most of the memset overhead appears to be coming from > >> InitCatCache. > > > >

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane wrote: > Robert Haas writes: >> Full results, and call graph, attached.  The first obvious fact is >> that most of the memset overhead appears to be coming from >> InitCatCache. > > AFAICT that must be the palloc0 calls that are zeroing out (mostly) > the

Re: [HACKERS] Per-column collation, work in progress

2010-11-24 Thread Peter Eisentraut
On ons, 2010-09-22 at 19:44 +0900, Itagaki Takahiro wrote: > * CREATE TABLE (LIKE table_with_collation) doesn't inherit collations. This was fixed in the CF2010-11 patch. > * psql \d needs a separator between collate and not null modifiers. And this as well. -- Sent via pgsql-hackers mailing

Re: [HACKERS] Per-column collation, work in progress

2010-11-24 Thread Peter Eisentraut
On tor, 2010-10-14 at 22:54 -0400, Robert Haas wrote: > It seems you've falsified the header comment in > pathkeys_useful_for_merging(), although I guess it's already false > because it doesn't seem to have been updated for the NULLS ASC/DESC > stuff, and the interior comment in right_merge_directi

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Marti Raudsepp
On Wed, Nov 24, 2010 at 21:52, Tom Lane wrote: > Bruce Momjian writes: >> Notice the to_date()'s were not converted to constants in EXPLAIN so >> they are evaluated for every row.  to_date() is marked STABLE. > No.  This is per expectation.  Only IMMUTABLE functions can be folded to > constants

Re: [HACKERS] Per-column collation

2010-11-24 Thread Peter Eisentraut
On mån, 2010-11-22 at 11:58 +0900, Itagaki Takahiro wrote: > * Did you see any performance regression by collation? > I found a bug in lc_collate_is_c(); result >= 0 should be > checked before any other checks. SearchSysCache1() here > would be a performance regression. That code turned out to be

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Tom Lane
Bruce Momjian writes: > I realize they can't be converted to constants before the query starts > but is there a reason we can't convert those functions to constants in > the executor before a table scan? Other than the significant number of cycles that would be wasted (in most cases) checking for

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > Full results, and call graph, attached. The first obvious fact is > that most of the memset overhead appears to be coming from > InitCatCache. AFAICT that must be the palloc0 calls that are zeroing out (mostly) the hash bucket headers. I don't see any real way to make that

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > Revised patch attached. The asserts in AtProcExit_LocalBuffers are a bit pointless since you forgot to remove the code that forcibly zeroes LocalRefCount[]... otherwise +1. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Extensions, this time with a patch

2010-11-24 Thread Dimitri Fontaine
Itagaki Takahiro writes: > RECOVERY_COMMAND_FILE is opened twice in the patch. The first time > is for checking the existence, and the second time is for parsing. > Instead of the repeat, how about adding FILE* version of parser? > It will be also called from ParseConfigFile() as a sub routine. >

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Gerhard Heift writes: > On Wed, Nov 24, 2010 at 01:20:36PM -0500, Robert Haas wrote: >> I tried configuring oprofile with --callgraph=10 and then running >> oprofile with -c, but it gives kooky looking output I can't interpret. > Have a look at the wiki: > http://wiki.postgresql.org/wiki/Profilin

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Notice the to_date()'s were not converted to constants in EXPLAIN so > > they are evaluated for every row. to_date() is marked STABLE. > > > Is this something we should improve? > > No. This is per expectation. Only IMMUTABLE functions can be folded

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Tom Lane
Bruce Momjian writes: > Notice the to_date()'s were not converted to constants in EXPLAIN so > they are evaluated for every row. to_date() is marked STABLE. > Is this something we should improve? No. This is per expectation. Only IMMUTABLE functions can be folded to constants in advance of th

[HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Bruce Momjian
Someone offlist reported query slowness because we don't convert function calls with all-constant parameters to be a constants before we start a sequential scan: EXPLAIN SELECT * FROM test WHERE x >= to_date('2001-01-01', '-MM-DD') AND x <= to_date('20

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 19:01:32 Robert Haas wrote: > Somehow I don't think I'm going to get much further with this without > figuring out how to get oprofile to cough up a call graph. I think to do that sensibly you need CFLAGS="-O2 -fno-omit-frame-pointer"... -- Sent via pgsql-hackers mai

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Gerhard Heift
On Wed, Nov 24, 2010 at 01:20:36PM -0500, Robert Haas wrote: > On Wed, Nov 24, 2010 at 1:06 PM, Tom Lane wrote: > > Robert Haas writes: > >> OK, patch attached. > > > > Two comments: > > Revised patch attached. > > I tried configuring oprofile with --callgraph=10 and then running > oprofile wit

Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-24 Thread Maciek Sakrejda
> Text format is marked as 0, binary format is 1. Sorry--I misread the docs. This is consistent and something does look fishy. Thanks for the tarball. I can take a look tonight. --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650) 242-35

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:06 PM, Tom Lane wrote: > Robert Haas writes: >> OK, patch attached. > > Two comments: Revised patch attached. I tried configuring oprofile with --callgraph=10 and then running oprofile with -c, but it gives kooky looking output I can't interpret. For example: 6

Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-24 Thread Radosław Smogura
I didn't described log correctly, 1st attached response is normal execution; flags QUERY_SUPPRESS_BEGIN | QUERY_ONESHOT, 2nd is compiled statement QUERY_SUPPRESS_BEGIN only. Text format is marked as 0, binary format is 1. The 1st shown execution (flags=17) is good, it tells that result is send

Re: [HACKERS] 8.4-vintage problem in postmaster.c

2010-11-24 Thread Stefan Kaltenbrunner
On 11/15/2010 03:24 PM, Alvaro Herrera wrote: Excerpts from Tom Lane's message of sáb nov 13 19:07:50 -0300 2010: Stefan Kaltenbrunner writes: On 11/13/2010 06:58 PM, Tom Lane wrote: Just looking at it, I think that the logic in canAcceptConnections got broken by somebody in 8.4, and then bro

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 10:33 AM, Tom Lane wrote: > Robert Haas writes: >> Right, that was my impression, too.  But, I think this may be partly a >> case of people talking past each other.  My impression of this >> conversation was a repetition of this sequence: > >> A: This syntax is bad. >> B:

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > OK, patch attached. Two comments: 1. A comment would help, something like "Assert we released all buffer pins". 2. AtProcExit_LocalBuffers should be redone the same way, for consistency (it likely won't make any performance difference). Note the comment for AtProcExit_Loca

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 11:33 AM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Nov 24, 2010 at 10:25 AM, Tom Lane wrote: >>> Or make it execute only in assert-enabled mode, perhaps. > >> But making the check execute only in assert-enabled more >> doesn't seem right, since the check actually

Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-24 Thread Maciek Sakrejda
>Result is oid=23, format=(0) T, value = 0x00,0x00,0x00,0x02 What do you mean regarding the format? Are you just inferring that from the data? If memory serves, the format of a particular column is not specified anywhere other than the RowDescription, and according to your JDBC log output above, t

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Pavel Stehule
Hello 2010/11/24 Tom Lane : > Robert Haas writes: >> Right, that was my impression, too.  But, I think this may be partly a >> case of people talking past each other.  My impression of this >> conversation was a repetition of this sequence: > >> A: This syntax is bad. >> B: But it's way faster! >

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 24, 2010 at 10:45 AM, Tom Lane wrote: >> It *will* be manifestly harder to use if users have to spell the >> argument types just so.  Consider int4 versus integer, varchar versus >> character varying (and not character varying(N)), etc etc.  I think >> that leavi

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 24, 2010 at 10:25 AM, Tom Lane wrote: >> Or make it execute only in assert-enabled mode, perhaps. > But making the check execute only in assert-enabled more > doesn't seem right, since the check actually acts to mask other coding > errors, rather than reveal the

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 10:25 AM, Tom Lane wrote: > Robert Haas writes: >> The first optimization that occurred to me was "remove the loop >> altogether". > > Or make it execute only in assert-enabled mode, perhaps. > > This check had some use back in the bad old days, but the ResourceOwner > mec

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 10:45 AM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Nov 24, 2010 at 9:52 AM, Tom Lane wrote: >>> Actually, what occurs to me to wonder is whether the facility has to be >>> guaranteed unique at all.  If for instance you have a group of overloaded >>> functions, is

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Joachim Wieland writes: > Whatever approach we use, we need to think about the use case where 1% > of the objects should be dumped but should also make sure that you can > more or less easily dump 99% of the objects. Roberts use case is the > 1% use case. Especially for the 99% case however, pg_du

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 24, 2010 at 9:52 AM, Tom Lane wrote: >> Actually, what occurs to me to wonder is whether the facility has to be >> guaranteed unique at all.  If for instance you have a group of overloaded >> functions, is there really a big use-case for dumping just one and not

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Dmitriy Igrishin writes: > I also want to propose to make it possible dump function definitions > as CREATE OR REPLACE FUNCTION rather than just CREATE > FUNCTION (as pg_dump dumps them now). It's intentional that pg_dump doesn't do that. Please don't think that pg_dump is a substitute for \ef.

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Tom Lane
Robert Haas writes: > Right, that was my impression, too. But, I think this may be partly a > case of people talking past each other. My impression of this > conversation was a repetition of this sequence: > A: This syntax is bad. > B: But it's way faster! > ...which makes no sense. However,

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Joachim Wieland
On Wed, Nov 24, 2010 at 9:38 AM, Andrew Dunstan wrote: > It would be unique, but a pain in the neck for users to get. Robert's idea > will have more traction with users. Whatever approach we use, we need to think about the use case where 1% of the objects should be dumped but should also make sur

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 24, 2010 at 2:10 AM, Heikki Linnakangas > wrote: >> Micro-optimizing that search for the non-zero value helps a little bit >> (attached). Reduces the percentage shown by oprofile from about 16% to 12% >> on my laptop. That "micro-optimization" looks to me like y

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Simon Riggs
On Wed, 2010-11-24 at 12:48 +0200, Heikki Linnakangas wrote: > When recovery starts, we fetch the oldestActiveXid from the checkpoint > record. Let's say that it's 100. We then start replaying WAL records > from the Redo pointer, and the first record (heap insert in your case) > contains an Xid th

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 9:52 AM, Tom Lane wrote: > Actually, what occurs to me to wonder is whether the facility has to be > guaranteed unique at all.  If for instance you have a group of overloaded > functions, is there really a big use-case for dumping just one and not > the whole group?  Even i

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Joachim Wieland writes: > On Wed, Nov 24, 2010 at 1:15 AM, Tom Lane wrote: >> Nope ... those strings are just helpful comments, they aren't really >> guaranteed to be unique identifiers.  In any case, it seems unlikely >> that a user could expect to get the more complicated cases exactly right >>

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Andrew Dunstan
On 11/24/2010 09:05 AM, Joachim Wieland wrote: On Wed, Nov 24, 2010 at 1:15 AM, Tom Lane wrote: Nope ... those strings are just helpful comments, they aren't really guaranteed to be unique identifiers. In any case, it seems unlikely that a user could expect to get the more complicated cases

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Joachim Wieland
On Wed, Nov 24, 2010 at 1:15 AM, Tom Lane wrote: > Nope ... those strings are just helpful comments, they aren't really > guaranteed to be unique identifiers.  In any case, it seems unlikely > that a user could expect to get the more complicated cases exactly right > other than by consulting "pg_d

[HACKERS] requested feature: limit for text or bytea parameters in log

2010-11-24 Thread Pavel Stehule
Hello We use a very rich log - for pgFouine processing. But sometime there are logger 1MB parameters. It's absolutely useless. These long values can be replaced by " first n chars ... truncated original length: 223636, md5: jhagjkafhskdjfhdsf Regards Pavel Stehule -- Sent via pgsql-hacker

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 8:41 AM, Andrew Dunstan wrote: > On 11/24/2010 07:29 AM, Robert Haas wrote: >>  As a first attempt at syntax, I might suggest something along the >> lines of "object type: object name", where the types and names might >> look to COMMENT ON for inspiration. > > pg_dump alrea

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Andrew Dunstan
On 11/24/2010 07:29 AM, Robert Haas wrote: As a first attempt at syntax, I might suggest something along the lines of "object type: object name", where the types and names might look to COMMENT ON for inspiration. pg_dump already uses a list of object types (e.g. as seen in the output fro

Re: [HACKERS] Extensions, this time with a patch

2010-11-24 Thread Itagaki Takahiro
On Tue, Nov 23, 2010 at 18:19, Dimitri Fontaine wrote: > Please find that done in attached v4 of the cfparser patch. RECOVERY_COMMAND_FILE is opened twice in the patch. The first time is for checking the existence, and the second time is for parsing. Instead of the repeat, how about adding FILE*

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Dmitriy Igrishin
Hey hackers, Completely agree with Robert ! It would be nice to dump functions definitions, e.g. to make it possible keep them in git separately. I also want to propose to make it possible dump function definitions as CREATE OR REPLACE FUNCTION rather than just CREATE FUNCTION (as pg_dump dumps th

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:06 AM, Tom Lane wrote: > =?ISO-8859-1?Q?C=E9dric_Villemain?= > writes: >> I think you (Robert) misunderstood dramatically what Pavel try to do. >> Pavel did an excellent optimization work for a specific point. This >> specific point looks crucial for me in the current b

Re: [HACKERS] Suggested "easy" TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:15 AM, Tom Lane wrote: > Josh Berkus writes: >>> Well, very little about pg_dump is very [E], IMNSHO. The question in my >>> mind here is what format the list file will take > >> I was thinking same format as pg_restore -l, only without the dumpIDs. > > Nope ... those st

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 2:10 AM, Heikki Linnakangas wrote: >> Anything we can do about this?  That's a lot of overhead, and it'd be >> a lot worse on a big machine with 8GB of shared_buffers. > > Micro-optimizing that search for the non-zero value helps a little bit > (attached). Reduces the perce

Re: [HACKERS] Instrument checkpoint sync calls

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:02 AM, Greg Smith wrote: > Robert Haas wrote: >> Did this get eaten by the email goblin, or you're still working on it? > > Fell behind due to an unfortunately timed bit of pneumonia.  Hurray for the > health benefits of cross country flights.  Will fix this up, rebase my

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Heikki Linnakangas
On 24.11.2010 13:38, Heikki Linnakangas wrote: It's dangerous to initialize latestObservedXid to anything to an older value. older value than the nextXid-1 from the checkpoint record, I meant to say. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hacker

  1   2   >