Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-20 Thread Marko Kreen
On Tue, Feb 21, 2012 at 02:11:35PM +0900, Kyotaro HORIGUCHI wrote: > I don't have any attachment to PQskipRemainingResults(), but I > think that some formal method to skip until Command-Complete('C') > without breaking session is necessary because current libpq does > so. We have such function: PQ

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-02-20 Thread Kyotaro HORIGUCHI
Hello, I don't have any attachment to PQskipRemainingResults(), but I think that some formal method to skip until Command-Complete('C') without breaking session is necessary because current libpq does so. > On Thu, Feb 16, 2012 at 02:24:19PM +0900, Kyotaro HORIGUCHI wrote: > > The choices o

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Bruce Momjian
On Mon, Feb 20, 2012 at 11:23:42PM +, Simon Riggs wrote: > > If you use pg_upgrade, and enable the checksum GUC, your database will > > become progressively checksummed, and as Simon pointed out, the only > > clean way is VACUUM FULL.  It is quite hard to estimate the checksum > > coverage of a

Re: [HACKERS] Displaying accumulated autovacuum cost

2012-02-20 Thread Fujii Masao
On Mon, Feb 20, 2012 at 11:00 PM, Robert Haas wrote: > On Mon, Feb 20, 2012 at 2:49 AM, Fujii Masao wrote: >> +DATA = pg_stat_statements--1.1.sql pg_stat_statements--1.0--1.1.sql \ >> +       pg_stat_statements--unpackaged--1.0.sql >> >> Though I'm not familiar with CREATE EXTENSION. Why did you

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-02-20 Thread Tom Lane
Peter Geoghegan writes: > Another look around shows that the CoerceToDomain struct takes its > location from the new Const location in turn, so my dirty little hack > will break the location of the CoerceToDomain struct, giving an > arguably incorrect caret position in some error messages. It woul

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-02-20 Thread Tom Lane
Peter Geoghegan writes: > Here is the single, hacky change I've made just for now to the core > parser to quickly see if it all works as expected: > *** transformTypeCast(ParseState *pstate, Ty > *** 2108,2113 > --- 2108,2116 > if (location < 0) > locatio

[HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-02-20 Thread Peter Geoghegan
On 20 February 2012 23:16, Peter Geoghegan wrote: > Clearly this change is a quick and dirty workaround, and something > better is required. The question I'd pose to the maintainer of this > code is: what business does the coerce_to_target_type call have > changing the location of the Const node r

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Simon Riggs
On Mon, Feb 20, 2012 at 11:09 PM, Bruce Momjian wrote: > On Sun, Feb 19, 2012 at 05:04:06PM -0500, Robert Haas wrote: >> Another disadvantage of the current scheme is that there's no >> particularly easy way to know that your whole cluster has checksums. >> No matter how we implement checksums, yo

[HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-02-20 Thread Peter Geoghegan
On 16 February 2012 21:11, Peter Geoghegan wrote: > *       # XXX: This test currently fails!: >        #verify_normalizes_correctly("SELECT cast('1' as dnotnull);","SELECT > cast(? as dnotnull);",conn, "domain literal canonicalization/cast") > > It appears to fail because the CoerceToDomain node

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Simon Riggs
On Mon, Feb 20, 2012 at 6:22 PM, Robert Haas wrote: > On Mon, Feb 20, 2012 at 12:53 PM, Josh Berkus wrote: >> On 2/20/12 5:57 AM, Robert Haas wrote: >>> 3. You're rearranging the page header in a way that I find ugly and baroque. >> >> Guys, are we talking about an on-disk format change?  If so,

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Bruce Momjian
On Sun, Feb 19, 2012 at 05:04:06PM -0500, Robert Haas wrote: > Another disadvantage of the current scheme is that there's no > particularly easy way to know that your whole cluster has checksums. > No matter how we implement checksums, you'll have to rewrite every > table in the cluster in order to

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-20 Thread Shigeru Hanada
2012/02/21 0:58 "Kevin Grittner" : > > "Albe Laurenz" wrote: > > > I read the example carefully, and it seems to me that it is > > necessary for the read-only transaction (T3)v to be SERIALIZABLE so > > that T1 is aborted and the state that T3 saw remains valid. > > Correct. Hm, agreed that isolat

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Josh Berkus
>> Guys, are we talking about an on-disk format change? If so, this may >> need to be kicked out of 9.2 ... > > Yes, we are. Simon's gone to some pains to make it backward > compatible, but IMHO it's a lot messier and less future-proof than it > could be with some more work, and if we commit th

Re: [HACKERS] Future of our regular expression code

2012-02-20 Thread Tom Lane
Billy Earney writes: > Thanks for your reply. So is the group leaning towards just maintaining > the current regex code base, or looking into introducing a new library > (RE2, PCRE, etc)? Or is this still open for discussion? Well, introducing a new library would create compatibility issues th

Re: [HACKERS] Future of our regular expression code

2012-02-20 Thread Billy Earney
Tom, Thanks for your reply. So is the group leaning towards just maintaining the current regex code base, or looking into introducing a new library (RE2, PCRE, etc)? Or is this still open for discussion? Thanks! Billy On Mon, Feb 20, 2012 at 3:35 PM, Tom Lane wrote: > Billy Earney writes:

Re: [HACKERS] Future of our regular expression code

2012-02-20 Thread Tom Lane
Billy Earney writes: > Also would it be possible to set a session variable (lets say PGREGEXTYPE) > and set it to ARE (current alg), RE2, or PCRE, that way users could choose > which implementation they want (unless we find a single implementation that > beats the others in almost all categories)

Re: [HACKERS] Future of our regular expression code

2012-02-20 Thread Billy Earney
Jay, Good links, and I've also looked at a few others with benchmarks. I believe most of the benchmarks are done before PCRE implemented jit. I haven't found a benchmark with jit enabled, so I'm not sure if it will make a difference. Also I'm not sure how accurately the benchmarks will show how

Re: [HACKERS] wal_buffers

2012-02-20 Thread Robert Haas
On Mon, Feb 20, 2012 at 3:59 AM, Simon Riggs wrote: >> There is no existing statistics view suitable to include such information. >> What about defining pg_stat_xlog or something? > > Perhaps pg_stat_perf so we don't need to find a new home every time. > > Thinking about it, I think renaming pg_st

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Robert Haas
On Mon, Feb 20, 2012 at 12:53 PM, Josh Berkus wrote: > On 2/20/12 5:57 AM, Robert Haas wrote: >> 3. You're rearranging the page header in a way that I find ugly and baroque. > > Guys, are we talking about an on-disk format change?  If so, this may > need to be kicked out of 9.2 ... Yes, we are.

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Josh Berkus
On 2/20/12 5:57 AM, Robert Haas wrote: > 3. You're rearranging the page header in a way that I find ugly and baroque. Guys, are we talking about an on-disk format change? If so, this may need to be kicked out of 9.2 ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via p

Re: [HACKERS] Incorrect behaviour when using a GiST index on points

2012-02-20 Thread Alexander Korotkov
On Mon, Feb 20, 2012 at 7:22 PM, Tom Lane wrote: > Alexander Korotkov writes: > > On Thu, Feb 16, 2012 at 11:43 AM, Alexander Korotkov > > wrote: > >> Described differences leads to incorrect behaviour of GiST index. > >> The question is: what is correct way to fix it? Should on_pb also use > FP

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-20 Thread Kevin Grittner
"Albe Laurenz" wrote: > I read the example carefully, and it seems to me that it is > necessary for the read-only transaction (T3) to be SERIALIZABLE so > that T1 is aborted and the state that T3 saw remains valid. Correct. > If I understand right, I agree with your correction. :-) -Kevi

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-20 Thread Albe Laurenz
Kevin Grittner wrote: > > If your query involves foreign scans on two foreign tables on the > > same foreign server, these should always see the same snapshot, > > because that's how it works with two scans in one query on local > > tables. > > That makes sense. > > So I think it should be REPEAT

Re: [HACKERS] REASSIGN OWNED lacks support for FDWs

2012-02-20 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 15, 2012 at 12:58 PM, Tom Lane wrote: >> As per http://archives.postgresql.org/pgsql-general/2012-02/msg00304.php >> there is no switch case in shdepReassignOwned for foreign data wrappers. >> >> The obvious short-term answer (and probably the only back-patchabl

Re: [HACKERS] REASSIGN OWNED lacks support for FDWs

2012-02-20 Thread Robert Haas
On Wed, Feb 15, 2012 at 12:58 PM, Tom Lane wrote: > As per http://archives.postgresql.org/pgsql-general/2012-02/msg00304.php > there is no switch case in shdepReassignOwned for foreign data wrappers. > > The obvious short-term answer (and probably the only back-patchable one) > is to add a case fo

Re: [HACKERS] Incorrect behaviour when using a GiST index on points

2012-02-20 Thread Tom Lane
Alexander Korotkov writes: > On Thu, Feb 16, 2012 at 11:43 AM, Alexander Korotkov > wrote: >> Described differences leads to incorrect behaviour of GiST index. >> The question is: what is correct way to fix it? Should on_pb also use FP* >> or consistent method should behave like on_pb? > Any comm

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-20 Thread Kevin Grittner
"Albe Laurenz" wrote: > If your query involves foreign scans on two foreign tables on the > same foreign server, these should always see the same snapshot, > because that's how it works with two scans in one query on local > tables. That makes sense. > So I think it should be REPEATABLE READ

Re: [HACKERS] [v9.2] Add GUC sepgsql.client_label

2012-02-20 Thread Yeb Havinga
On 2012-02-05 10:09, Kohei KaiGai wrote: The attached part-1 patch moves related routines from hooks.c to label.c because of references to static variables. The part-2 patch implements above mechanism. I took a short look at this patch but am stuck getting the regression test to run properly

Re: [HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-20 Thread Marc Mamin
> Hi Marc, > > Do you happen to know in which function, the extra time for the toast > storage is spent -- zlib compression? I saw a mention of the LZ4 > compression > algorithm that is BSD licensed as a Google summer of code project: > > http://code.google.com/p/lz4/ > > that compresses at almo

Re: [HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-20 Thread k...@rice.edu
On Mon, Feb 20, 2012 at 10:18:31AM +0100, Marc Mamin wrote: > > I looked into the complaint here of poor estimation for GIN > indexscans: > > http://archives.postgresql.org/pgsql-performance/2012-02/msg00028.php > > At first glance it sounds like a mistake in selectivity estimation, > > but it isn'

Re: [HACKERS] array_to_json re-encodes ARRAY of json type

2012-02-20 Thread Andrew Dunstan
On 02/20/2012 07:30 AM, Itagaki Takahiro wrote: If we pass an ARRAY of json type to array_to_json() function, the function seems to re-encode the JSON text. But should the following examples be the same result? I'm not sure why we don't have a special case for json type in datum_to_json() -- do

Re: [HACKERS] Displaying accumulated autovacuum cost

2012-02-20 Thread Robert Haas
On Mon, Feb 20, 2012 at 2:49 AM, Fujii Masao wrote: > In pg_stat_statements--1.0--1.1.sql, we should complain if script is sourced > in psql, as follows? > >    \echo Use "ALTER EXTENSION pg_stat_statements UPDATE TO '1.1'" to > load this file. \quit Yeah, maybe. I don't know if we want to put t

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Robert Haas
On Mon, Feb 20, 2012 at 4:18 AM, Simon Riggs wrote: > What straightforward implementation is that?? This *is* the straightforward > one. > > God knows what else we'd break if we drop the lock, reacquire as an > exclusive, then drop it again and reacquire in shared mode. Code tends > to assume tha

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-20 Thread Robert Haas
On Mon, Feb 20, 2012 at 5:05 AM, Jan Urbański wrote: > On 20/02/12 04:29, Tom Lane wrote: >> =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: On 18/02/12 21:17, Tom Lane wrote: > Dave Malcolm at Red Hat has been working on a static code analysis tool > for Python-related C code.  He reports

[HACKERS] array_to_json re-encodes ARRAY of json type

2012-02-20 Thread Itagaki Takahiro
If we pass an ARRAY of json type to array_to_json() function, the function seems to re-encode the JSON text. But should the following examples be the same result? I'm not sure why we don't have a special case for json type in datum_to_json() -- do we need to pass-through json types in it? =# \x =#

Re: [HACKERS] Potential reference miscounts and segfaults in plpython.c

2012-02-20 Thread Jan Urbański
On 20/02/12 04:29, Tom Lane wrote: > =?UTF-8?B?SmFuIFVyYmHFhHNraQ==?= writes: >>> On 18/02/12 21:17, Tom Lane wrote: Dave Malcolm at Red Hat has been working on a static code analysis tool for Python-related C code. He reports here on some preliminary results for plpython.c: h

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-20 Thread Simon Riggs
On Mon, Feb 20, 2012 at 12:42 AM, Robert Haas wrote: > I think we could do worse than this patch, but I don't really believe > it's ready for commit.  We don't have a single performance number > showing how much of a performance regression this causes, either on > the master or on the standby, on

Re: [HACKERS] Qual evaluation cost estimates for GIN indexes

2012-02-20 Thread Marc Mamin
> I looked into the complaint here of poor estimation for GIN indexscans: > http://archives.postgresql.org/pgsql-performance/2012-02/msg00028.php > At first glance it sounds like a mistake in selectivity estimation, > but it isn't: the rowcount estimates are pretty nearly dead on. > The problem is

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-20 Thread Albe Laurenz
I wrote: > Shigeru Hanada wrote: >>> - Since a rescan is done by rewinding the cursor, is it necessary >>>to have any other remote isolation level than READ COMMITED? >>>There is only one query issued per transaction. >> >> If multiple foreign tables on a foreign server is used in a local q

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-02-20 Thread Fujii Masao
On Sun, Feb 19, 2012 at 3:01 AM, Jeff Janes wrote: > I've tested your v9 patch.  I no longer see any inconsistencies or > lost transactions in the recovered database.  But occasionally I get > databases that fail to recover at all. > It has always been with the exact same failed assertion, at xlog

Re: [HACKERS] Google Summer of Code? Call for mentors.

2012-02-20 Thread Jehan-Guillaume (ioguix) de Rorthais
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, On 01-02-2012 20:59, Josh Berkus wrote: > Hackers, > > The call is now open for Google Summer of Code. > > If you are interested in being a GSoC mentor this summer, please reply > to this email. I want to gauge whether or not we should parti

Re: [HACKERS] wal_buffers

2012-02-20 Thread Simon Riggs
On Mon, Feb 20, 2012 at 4:10 AM, Fujii Masao wrote: > On Mon, Feb 20, 2012 at 3:08 AM, Robert Haas wrote: >> On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira >> wrote: >>> On 19-02-2012 02:24, Robert Haas wrote: I have attached tps scatterplots.  The obvious conclusion appears to

Re: [HACKERS] leakproof

2012-02-20 Thread Albe Laurenz
Greg Stark wrote: > I suspect this is wrong for similar reasons as "pure" but I'll throw > it out there: "hermetic" I personally have no problem with "leakproof", but if it does not tickle the right associations with many people: What about "secure"? It is also not self-explanatory, but it might

Re: [HACKERS] leakproof

2012-02-20 Thread Yeb Havinga
On 2012-02-20 06:37, Don Baccus wrote: On Feb 19, 2012, at 7:24 PM, Tom Lane wrote: It's not clear to me whether pure/leakproof functions are meant to be a strict subset of immutable functions Superset, not subset, unless my guessing is wrong. How could "pure" be a subset of "immutable"? If

Re: [HACKERS] Bugs/slowness inserting and indexing cubes

2012-02-20 Thread Alexander Korotkov
On Wed, Feb 15, 2012 at 7:28 PM, Tom Lane wrote: > Alexander Korotkov writes: > > On Wed, Feb 15, 2012 at 4:26 PM, Heikki Linnakangas < > > heikki.linnakan...@enterprisedb.com> wrote: > >> So, I think we should go with your original fix and simply do nothing in > >> gistRelocateBuildBuffersOnSpl

Re: [HACKERS] Google Summer of Code? Call for mentors.

2012-02-20 Thread Alexander Korotkov
On Thu, Feb 16, 2012 at 12:56 PM, Dave Page wrote: > On Wed, Feb 15, 2012 at 10:18 PM, Josh Berkus wrote: > > Hackers, > > > > The call is now open for Google Summer of Code. > > > > If you are interested in being a GSoC mentor this summer, please reply > > to this email. I want to gauge whethe

Re: [HACKERS] Incorrect behaviour when using a GiST index on points

2012-02-20 Thread Alexander Korotkov
On Thu, Feb 16, 2012 at 11:43 AM, Alexander Korotkov wrote: > Described differences leads to incorrect behaviour of GiST index. > The question is: what is correct way to fix it? Should on_pb also use FP* > or consistent method should behave like on_pb? > Any comments on this? Current behaviour de