Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 10:37 PM, Daniel Farina wrote: > I added programming around various NULL returns reading GUCs in this > revision, v4. Okay, one more of those fridge-logic bugs. Sorry for the noise. v5. A missing PG_RETHROW to get the properly finally-esque semantics: --- a/contrib/dbli

Re: [HACKERS] citext like searches using index

2013-03-19 Thread David E. Wheeler
On Mar 17, 2013, at 6:35 AM, Thorbjørn Weidemann wrote: > Hi David, > > I found your email-address on > http://www.postgresql.org/docs/9.2/static/citext.html. I hope it's ok to > contact you this way. > I would like to thank you for taking the time to make citext available for > Postgres, an

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 6:10 PM, Daniel Farina wrote: > On Tue, Mar 19, 2013 at 3:06 PM, Daniel Farina wrote: >> On Tue, Mar 19, 2013 at 2:41 PM, Tom Lane wrote: >>> Daniel Farina writes: On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane wrote: > I'd be inclined to eat the cost of calling PQp

Re: [HACKERS] machine-parseable object descriptions

2013-03-19 Thread Alvaro Herrera
Alvaro Herrera wrote: > .. and here's the patch. I forgot an example involving the funniest of all object classes: default ACLs. Here it is. alvherre=# create role rol1; CREATE ROLE alvherre=# create role rol2; CREATE ROLE alvherre=# create schema rol1 authorization rol1; CREATE SCHEMA alvherre=

Re: [HACKERS] machine-parseable object descriptions

2013-03-19 Thread Alvaro Herrera
.. and here's the patch. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services *** a/src/backend/catalog/dependency.c --- b/src/backend/catalog/dependency.c *** *** 67,72 --- 67,73 #include "commands/trigger.

Re: [HACKERS] machine-parseable object descriptions

2013-03-19 Thread Alvaro Herrera
Dimitri Fontaine wrote: > Tom Lane writes: > > I could also live with keeping the schema column as proposed, if people > > think it has a use, but letting it be redundant with a schema name > > included in the identity string. But it seems like a bad idea to try to > > shear schema off of identit

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
On 3/19/13 10:05 PM, Tom Lane wrote: FWIW, I would argue that any tradeoffs we make in this area must be made on the assumption of no such acceleration. If we can later make things better for Intel(TM) users, that's cool, but let's not screw those using other CPUs. I see compatibility with the

[HACKERS] Ignore invalid indexes in pg_dump

2013-03-19 Thread Michael Paquier
Hi, If failures happen with CREATE INDEX CONCURRENTLY, the system will be let with invalid indexes. I don't think that the user would like to see invalid indexes of an existing system being recreated as valid after a restore. So why not removing from a dump invalid indexes with something like the

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Tom Lane
Greg Smith writes: > While being a lazy researcher today instead of writing code, I > discovered that the PNG file format includes a CRC-32 on its data > chunks, and to support that there's a CRC32 function inside of zlib: > http://www.zlib.net/zlib_tech.html Hah, old sins keep coming back to

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 3:06 PM, Daniel Farina wrote: > On Tue, Mar 19, 2013 at 2:41 PM, Tom Lane wrote: >> Daniel Farina writes: >>> On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane wrote: I'd be inclined to eat the cost of calling PQparameterStatus every time (which won't be that much) and

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
On 3/19/13 8:17 PM, Simon Riggs wrote: We know that will work, has reasonable distribution characteristics and might even speed things up rather than have two versions of CRC in the CPU cache. That sounds reasonable to me. All of these CRC options have space/time trade-offs via how large the

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Simon Riggs
On 20 March 2013 00:03, Greg Smith wrote: > Simon suggested the other day that we should make the > exact checksum mechanism used pluggable at initdb time, just some last > minute alternatives checking on the performance of the real server code. > I've now got the WAL CRC32, the zlib CRC32, and t

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Ants Aasma
On Wed, Mar 20, 2013 at 12:52 AM, Greg Smith wrote: > On 3/19/13 6:08 PM, Ants Aasma wrote: >> >> My main worry is that there is a reasonably >> large population of users out there that don't have that acceleration >> capability and will have to settle for performance overhead 4x worse >> than wha

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
On 3/19/13 7:13 PM, Daniel Farina wrote: I'm confused. Postgres includes a CRC32 implementation for WAL, does it not? Are you referring to something else? I'm just pointing out that zlib includes one, too, and they might be more motivated/able as a project to chase after Intel's hardware acc

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Andrew Dunstan
On 03/19/2013 06:52 PM, Greg Smith wrote: While being a lazy researcher today instead of writing code, I discovered that the PNG file format includes a CRC-32 on its data chunks, and to support that there's a CRC32 function inside of zlib: http://www.zlib.net/zlib_tech.html Is there anywh

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 3:52 PM, Greg Smith wrote: > On 3/19/13 6:08 PM, Ants Aasma wrote: >> >> My main worry is that there is a reasonably >> large population of users out there that don't have that acceleration >> capability and will have to settle for performance overhead 4x worse >> than what

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
On 3/19/13 6:08 PM, Ants Aasma wrote: My main worry is that there is a reasonably large population of users out there that don't have that acceleration capability and will have to settle for performance overhead 4x worse than what you currently measured for a shared buffer swapping workload. Th

[HACKERS] Add regression tests for ROLE (USER)

2013-03-19 Thread Robins Tharakan
Hi, Please find attached a patch to take 'make check' code-coverage of ROLE (USER) from 59% to 91%. Any feedback is more than welcome. -- Robins Tharakan regress_user.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Ants Aasma
On Tue, Mar 19, 2013 at 11:28 PM, Greg Smith wrote: > I don't remember if there's any good precedent for whether this form of BSD > licensed code can be assimilated into PostgreSQL without having to give > credit to Intel in impractical places. I hate these licenses with the > binary restrictions

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 2:41 PM, Tom Lane wrote: > Daniel Farina writes: >> On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane wrote: >>> I'd be inclined to eat the cost of calling PQparameterStatus every time >>> (which won't be that much) and instead try to optimize by avoiding the >>> GUC-setting over

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Tom Lane
Daniel Farina writes: > On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane wrote: >> I'd be inclined to eat the cost of calling PQparameterStatus every time >> (which won't be that much) and instead try to optimize by avoiding the >> GUC-setting overhead if the current value matches the local setting. >>

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
On 3/18/13 8:17 PM, Ants Aasma wrote: I looked for fast CRC implementations on the net. The fastest plain C variant I could find was one produced by Intels R&D department (available with a BSD license [1], requires some porting). Very specifically, it references http://opensource.org/licenses/

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Tue, Mar 19, 2013 at 1:16 PM, Tom Lane wrote: > Daniel Farina writes: >> Similar in purpose to cc3f281ffb0a5d9b187e7a7b7de4a045809ff683, but >> taking into account that a dblink caller may choose to cause arbitrary >> changes to DateStyle and IntervalStyle. To handle this, it is >> necessary

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Tom Lane
Daniel Farina writes: > Similar in purpose to cc3f281ffb0a5d9b187e7a7b7de4a045809ff683, but > taking into account that a dblink caller may choose to cause arbitrary > changes to DateStyle and IntervalStyle. To handle this, it is > necessary to use PQparameterStatus before parsing any input, every

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Greg Smith
On 3/8/13 4:40 PM, Greg Stark wrote: On Fri, Mar 8, 2013 at 5:46 PM, Josh Berkus wrote: After some examination of the systems involved, we conculded that the issue was the FreeBSD drivers for the new storage, which were unstable and had custom source patches. However, without PostgreSQL checks

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Tom Lane
Jeff Davis writes: > I will move back to verifying the page hole, as well. > There are a few approaches: > 1. Verify that the page hole is zero before write and after read. > 2. Include it in the calculation (if we think there are some corner > cases where the hole might not be all zero). > 3. Z

Re: [HACKERS] Improving avg performance for numeric

2013-03-19 Thread Tom Lane
I wrote: > [ looks at patch... ] Oh, I see what's affecting the plan: you changed > the aggtranstypes to internal for a bunch of aggregates. That's not > very good, because right now the planner takes that to mean that the > aggregate could eat a lot of space. We don't want that to happen for >

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Simon Riggs
On 19 March 2013 00:17, Ants Aasma wrote: > I looked for fast CRC implementations on the net. Thanks very much for great input. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Simon Riggs
On 19 March 2013 17:18, Jeff Davis wrote: > I will move back to verifying the page hole, as well. That was agreed long ago... > There are a few approaches: > > 1. Verify that the page hole is zero before write and after read. > 2. Include it in the calculation (if we think there are some corner

Re: [HACKERS] [pgsql-advocacy] Call for Google Summer of Code mentors, admins

2013-03-19 Thread Thom Brown
On 14 February 2013 18:02, Josh Berkus wrote: > Folks, > > Once again, Google is holding Summer of Code. We need to assess whether > we want to participate this year. > > Questions: > > - Who wants to mentor for GSOC? > > - Who can admin for GSOC? Thom? > > - Please suggest project ideas for GSO

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Jeff Davis
On Fri, 2013-03-15 at 14:32 +0200, Ants Aasma wrote: > The most obvious case here is that you > can swap any number of bytes from 0x00 to 0xFF or back without > affecting the hash. That's a good point. Someone (Simon?) had brought that up before, but you and Tom convinced me that it's a problem. A

Re: [HACKERS] Enabling Checksums

2013-03-19 Thread Jeff Davis
On Sat, 2013-03-16 at 20:41 -0400, Tom Lane wrote: > Simon Riggs writes: > > On 15 March 2013 13:08, Andres Freund wrote: > >> I commented on this before, I personally think this property makes > >> fletcher a > >> not so good fit for this. Its not uncommon for parts of a block being > >> all-z

Re: [HACKERS] Improving avg performance for numeric

2013-03-19 Thread Tom Lane
[ please do not top-reply ] Hadi Moshayedi writes: > On Tue, Mar 19, 2013 at 6:25 PM, Tom Lane wrote: >> Uh, what? Fooling around with the implementation of avg() should surely >> not change any planning decisions. > I am not sure how this works, but I also changed numeric sum(), and the > vie

Re: [HACKERS] Improving avg performance for numeric

2013-03-19 Thread Hadi Moshayedi
I am not sure how this works, but I also changed numeric sum(), and the views in question had a numeric sum() column. Can that have any impact? I am going to dig deeper to see why this happens. On Tue, Mar 19, 2013 at 6:25 PM, Tom Lane wrote: > Kevin Grittner writes: > > Hadi Moshayedi wrote:

Re: [HACKERS] Improving avg performance for numeric

2013-03-19 Thread Tom Lane
Kevin Grittner writes: > Hadi Moshayedi wrote: >> I also noticed that this patch makes matview test fail. It seems >> that it just changes the ordering of rows for queries like >> "SELECT * FROM tv;". Does this seem like a bug in my patch, or >> should we add "ORDER BY" clauses to this test to ma

Re: [HACKERS] Improving avg performance for numeric

2013-03-19 Thread Pavel Stehule
2013/3/19 Kevin Grittner : > Hadi Moshayedi wrote: > >> I updated the patch by taking ideas from your patch, and unifying >> the transition struct and update function for different >> aggregates. The speed of avg improved even more. It now has 60% >> better performance than the current committed v

Re: [HACKERS] Improving avg performance for numeric

2013-03-19 Thread Kevin Grittner
Hadi Moshayedi wrote: > I updated the patch by taking ideas from your patch, and unifying > the transition struct and update function for different > aggregates. The speed of avg improved even more. It now has 60% > better performance than the current committed version. Outstanding! > I also no

Re: [HACKERS] Writable FDW: returning clauses.

2013-03-19 Thread Tom Lane
Ronan Dunklau writes: > While implementing the new writable API for FDWs, I wondered wether they > are any obvious problems with the following behavior for handling returning > clauses (for the delete case). > The goal is to fetch all required attributes during the initial scan, and > avoid fe

Re: [HACKERS] pg_upgrade segfaults when given an invalid PGSERVICE value

2013-03-19 Thread Steve Singer
On 13-03-18 09:17 PM, Bruce Momjian wrote: On Mon, Mar 18, 2013 at 12:08:09PM -0400, Steve Singer wrote: If you try running pg_upgrade with the PGSERVICE environment variable set to some invalid/non-existent service pg_upgrade segfaults Program received signal SIGSEGV, Segmentation fault. 0x000

Re: [HACKERS] Trust intermediate CA for client certificates

2013-03-19 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > As far as I'm concerned that's the immediate problem fixed. It may be > worth adding a warning on startup if we find non-self-signed certs in > root.crt too, something like 'WARNING: Intermediate certificate found in > root.crt. This does not do what

Re: [HACKERS] Trust intermediate CA for client certificates

2013-03-19 Thread Bruce Momjian
On Tue, Mar 19, 2013 at 09:37:18PM +0800, Craig Ringer wrote: > On 03/19/2013 08:39 PM, Stephen Frost wrote: > > Craig, > > > > * Craig Ringer (cr...@2ndquadrant.com) wrote: > >> Yep, in most applications I've seen you usually store a list of > >> authorized SubjectDNs or you just use your own self

Re: [HACKERS] Trust intermediate CA for client certificates

2013-03-19 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/19/2013 08:39 PM, Stephen Frost wrote: > Craig, > > * Craig Ringer (cr...@2ndquadrant.com) wrote: >> Yep, in most applications I've seen you usually store a list of >> authorized SubjectDNs or you just use your own self-signed root and >> issue

Re: postgres_fdw vs data formatting GUCs (was Re: [HACKERS] [v9.3] writable foreign tables)

2013-03-19 Thread Daniel Farina
On Thu, Mar 14, 2013 at 8:07 PM, Tom Lane wrote: > Daniel Farina writes: >> On Tue, Mar 12, 2013 at 11:51 AM, Tom Lane wrote: >>> Yeah, watching the remote side's datestyle and intervalstyle and >>> matching them (for both input and output) would probably work. > >> Alright, so I've been whackin

Re: [HACKERS] Trust intermediate CA for client certificates

2013-03-19 Thread Stephen Frost
Craig, * Craig Ringer (cr...@2ndquadrant.com) wrote: > Yep, in most applications I've seen you usually store a list of > authorized SubjectDNs or you just use your own self-signed root and > issue certs from it. Even with a self-signed root issuing certs, you need to map the individual cert to a

Re: [GENERAL] [HACKERS] Trust intermediate CA for client certificates

2013-03-19 Thread Bruce Momjian
On Tue, Mar 19, 2013 at 01:46:32AM -0400, Stephen Frost wrote: > > I guess that suggests we should be calling this something like > > 'ssl_authorized_client_roots'. > > I'm no longer convinced that this really makes sense and I'm a bit > worried about the simple authentication issue which I though

Re: [HACKERS] backward incompatible pg_basebackup and pg_receivexlog

2013-03-19 Thread Magnus Hagander
On Tue, Mar 19, 2013 at 11:39 AM, Heikki Linnakangas wrote: > On 19.03.2013 04:42, Peter Eisentraut wrote: >> >> pg_basebackup and pg_receivexlog from 9.3 won't work with earlier >> servers anymore. I wonder if this has been fully thought through. We >> have put in a lot of effort to make client

Re: [HACKERS] backward incompatible pg_basebackup and pg_receivexlog

2013-03-19 Thread Heikki Linnakangas
On 19.03.2013 04:42, Peter Eisentraut wrote: pg_basebackup and pg_receivexlog from 9.3 won't work with earlier servers anymore. I wonder if this has been fully thought through. We have put in a lot of effort to make client programs compatible with many server versions as well as keeping the cli

[HACKERS] Writable FDW: returning clauses.

2013-03-19 Thread Ronan Dunklau
Hello. While implementing the new writable API for FDWs, I wondered wether they are any obvious problems with the following behavior for handling returning clauses (for the delete case). The goal is to fetch all required attributes during the initial scan, and avoid fetching data on the delete

Re: [HACKERS] Identity projection

2013-03-19 Thread Kyotaro HORIGUCHI
Thank you to all involved. > On Friday, March 15, 2013 12:52 AM Tom Lane wrote: > > I wrote: > > > ... So I think this patch is missing a bet by not > > > accepting equal() expressions. > > > > I've committed this with that logic, a comment explaining exactly why > > this is the way to do it, and

Re: [HACKERS] Re: proposal: a width specification for s specifier (format function), fix behave when positional and ordered placeholders are used

2013-03-19 Thread Kyotaro HORIGUCHI
Thank you for committing this patch. > Applied with some mostly-cosmetic adjustments. I also took the > liberty of changing some of the error message texts to line up > more closely with the expanded documentation (eg, use "format > specifier" not "conversion specifier" because that's the phrase