Re: [HACKERS] Allow peer/ident to fall back to md5?

2014-10-28 Thread Noah Misch
On Wed, Oct 29, 2014 at 10:52:38AM +0800, Craig Ringer wrote: > On 10/29/2014 10:45 AM, Tom Lane wrote: > > Craig Ringer writes: > >> At pgconf-eu Álvaro and I were discussing the idea of allowing 'peer' > >> and 'ident' authentication to fall back to md5 if the peer/ident check > >> failed. > >

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-10-28 Thread Amit Kapila
On Tue, Oct 28, 2014 at 9:27 AM, Dilip kumar wrote: > On 28 October 2014 09:18, Amit Kapila Wrote, > > >I am worried about the case if after setting the inAbort flag, > > >PQCancel() fails (returns error). > > > > > >> If select(maxFd + 1, workerset, NULL, NULL, &tv); come out, we can know whether

Re: [HACKERS] pg_dump/pg_restore seem broken on hamerkop

2014-10-28 Thread Tom Lane
I wrote: > Alvaro Herrera writes: >> [Some more code and git-log reading later] I see that the %z is a very >> recent addition: it only got there as of commit ad5d46a449, of September >> 5th ... and now I also see that hamerkop's last green run before the >> failure, on Oct 13rd, did *not* includ

Re: [HACKERS] Allow peer/ident to fall back to md5?

2014-10-28 Thread Craig Ringer
On 10/29/2014 10:45 AM, Tom Lane wrote: > Craig Ringer writes: >> At pgconf-eu Álvaro and I were discussing the idea of allowing 'peer' >> and 'ident' authentication to fall back to md5 if the peer/ident check >> failed. > > I think it would be acceptable to define *new* auth modes that work > th

Re: [HACKERS] Allow peer/ident to fall back to md5?

2014-10-28 Thread Tom Lane
Craig Ringer writes: > At pgconf-eu Álvaro and I were discussing the idea of allowing 'peer' > and 'ident' authentication to fall back to md5 if the peer/ident check > failed. I think it would be acceptable to define *new* auth modes that work that way. I'm violently against redefining the meani

Re: [HACKERS] logical decoding - reading a user catalog table

2014-10-28 Thread Steve Singer
On 10/28/2014 01:31 PM, Andres Freund wrote: On 2014-10-25 18:18:07 -0400, Steve Singer wrote: My logical decoding plugin is occasionally getting this error "could not resolve cmin/cmax of catalog tuple" I get this when my output plugin is trying to read one of the user defined catalog tables

[HACKERS] Allow peer/ident to fall back to md5?

2014-10-28 Thread Craig Ringer
Hi all At pgconf-eu Álvaro and I were discussing the idea of allowing 'peer' and 'ident' authentication to fall back to md5 if the peer/ident check failed. This can be done backwards compatibly and without protocol changes. A client won't be able to tell that the pg_hba.conf line wasn't 'md5'.

Re: [HACKERS] how to handle missing "prove"

2014-10-28 Thread Andrew Dunstan
On 10/28/2014 09:16 PM, Tom Lane wrote: Peter Eisentraut writes: Here is a patch to use "missing" to handle the case when "prove" is not present. Wouldn't it be easier to do what we do for Perl, viz in Makefile.global.in ifneq (@PERL@,) # quoted to protect pathname with spaces PERL

Re: [HACKERS] how to handle missing "prove"

2014-10-28 Thread Peter Eisentraut
On 10/28/14 9:16 PM, Tom Lane wrote: > Peter Eisentraut writes: >> Here is a patch to use "missing" to handle the case when "prove" is not >> present. > > Wouldn't it be easier to do what we do for Perl, viz in Makefile.global.in > > ifneq (@PERL@,) > # quoted to protect pathname with spaces

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Peter Eisentraut
On 10/27/14 7:36 PM, Stephen Frost wrote: > MySQL: > http://dev.mysql.com/doc/refman/5.1/en/privileges-provided.html#priv_file > > (note they provide a way to limit access also, via secure_file_priv) They have a single privilege to allow the user to read or write any file. I think that feature c

Re: [HACKERS] how to handle missing "prove"

2014-10-28 Thread Tom Lane
Peter Eisentraut writes: > Here is a patch to use "missing" to handle the case when "prove" is not > present. Wouldn't it be easier to do what we do for Perl, viz in Makefile.global.in ifneq (@PERL@,) # quoted to protect pathname with spaces PERL= '@PERL@' else PERL

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-28 Thread Peter Eisentraut
On 10/28/14 12:46 AM, Noah Misch wrote: > Agreed. Having this framework when the pg_upgrade test suite originated would > have prevented acquiring parallel implementations in Perl and shell. Yes, that was certainly a motivation, and I would like to continue work on pg_upgrade testing. > Concrete

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-28 Thread Peter Eisentraut
On 10/27/14 11:41 AM, Robert Haas wrote: > Beyond all that, I have serious doubts about whether, even if we > eventually get these tests mostly working in most places, whether they > will actually catch any bugs. Well, they caught the fact that pg_basebackup can't back up tablespaces with names lo

Re: [HACKERS] pg_basebackup fails with long tablespace paths

2014-10-28 Thread Peter Eisentraut
On 10/20/14 2:59 PM, Tom Lane wrote: > My Salesforce colleague Thomas Fanghaenel observed that the TAP tests > for pg_basebackup fail when run in a sufficiently deeply-nested directory > tree. As for the test, we can do something like the attached to mark the test as "TODO". diff --git a/src/bin/

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-28 Thread Peter Eisentraut
On 10/7/14 1:57 PM, Tom Lane wrote: > Peter had a patch to eliminate the overhead of multiple subinstalls; > not sure where that stands, but presumably it would address your issue. It will need some cleverness to sort out the parallel make issues that were brought up in the review thread. -- Se

Re: [HACKERS] Portability issues in TAP tests

2014-10-28 Thread Peter Eisentraut
On 7/21/14 10:06 AM, Christoph Berg wrote: > 6. The tests fail if your $LANG isn't en_something: This was fixed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Portability issues in TAP tests

2014-10-28 Thread Peter Eisentraut
On 7/17/14 3:31 PM, Tom Lane wrote: > My Salesforce colleagues have been complaining that the TAP tests added > in 9.4 don't work terribly well for them. I've been poking at this, > and I believe this is a reasonably complete list of the problems: Quick followup: > 1. "make [install]check-world"

[HACKERS] how to handle missing "prove"

2014-10-28 Thread Peter Eisentraut
Here is a patch to use "missing" to handle the case when "prove" is not present. Other ideas? diff --git a/src/Makefile.global.in b/src/Makefile.global.in index b04d005..aff9af7 100644 --- a/src/Makefile.global.in +++ b/src/Makefile.global.in @@ -311,13 +311,13 @@ $(if $(filter $(PORTNAME),darwin)

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-28 Thread David Johnston
On Fri, Oct 24, 2014 at 6:36 AM, Alex Goncharov < alex.goncharov@gmail.com> wrote: > On Tue, Oct 21, 2014 at 10:16 AM, Tom Lane wrote: > >> (Of course, I'm not for the feature w.r.t. SQL either. But breaking data >> compatibility is just adding an entire new dimension of trouble. >> > > Anot

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-28 Thread Tom Lane
Jim Nasby writes: > On 10/28/14, 4:25 PM, David E. Wheeler wrote: >> This one, however, is more a judgment of people and their practices rather >> than the feature itself. Color me unimpressed. > +1. > Having users sweat of comma placement in this day and age is pretty stupid. I > can understa

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 7:26 PM, Robert Haas wrote: > On Tue, Oct 28, 2014 at 7:22 PM, Jim Nasby wrote: >> On 10/28/14, 3:48 PM, Simon Riggs wrote: >>> Given your description of pg_background it looks an awful lot like >>> infrastructure to make Autonomous Transactions work, but it doesn't >>> ev

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 7:22 PM, Jim Nasby wrote: > On 10/28/14, 3:48 PM, Simon Riggs wrote: >> Given your description of pg_background it looks an awful lot like >> infrastructure to make Autonomous Transactions work, but it doesn't >> even do that. I guess it could do in a very small additional

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Andres Freund
On 2014-10-28 20:17:57 +, Simon Riggs wrote: > On 28 October 2014 17:47, Andres Freund wrote: > > On 2014-10-28 17:45:36 +, Simon Riggs wrote: > >> I'd like to avoid all of the pain by making persistent AMs that are > >> recoverable after a crash, rather than during crash recovery. > > > >

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 4:48 PM, Simon Riggs wrote: > On 16 October 2014 16:22, Robert Haas wrote: >>> Might I gently enquire what the "something usable" we are going to see >>> in this release? I'm not up on current plans. >> >> I don't know how far I'm going to get for this release yet. I thin

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-28 Thread Jim Nasby
On 10/28/14, 3:48 PM, Simon Riggs wrote: Given your description of pg_background it looks an awful lot like infrastructure to make Autonomous Transactions work, but it doesn't even do that. I guess it could do in a very small additional patch, so maybe it is useful for something. What do you se

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-28 Thread Jim Nasby
On 10/28/14, 4:25 PM, David E. Wheeler wrote: This is a misfeature for the benefit of edit-lazy users only. This one, however, is more a judgment of people and their practices rather than the feature itself. Color me unimpressed. +1. Having users sweat of comma placement in this day and age

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 3:19 PM, Stephen Frost wrote: >> To articular my own concerns perhaps a bit better, there are two major >> things I don't like about the whole DIRALIAS proposal. Number one, >> you're creating this SQL object whose name is not actually used for >> anything other than manip

Re: [HACKERS] foreign data wrapper option manipulation during Create foreign table time?

2014-10-28 Thread Demai Ni
hi, Andrew, thanks for the quick response. M understanding is that Alter Foreign Table can change the option values by user. What I need is to change the value programmatic inside foreign data wrapper code, and hope someone already done so I can learn from the existing design. I thought this ema

Re: [HACKERS] foreign data wrapper option manipulation during Create foreign table time?

2014-10-28 Thread Andrew Dunstan
On 10/28/2014 05:26 PM, Demai Ni wrote: hi, guys, I am looking for a couple pointers here about fdw, and how to change the option values during CREATE table time. I am using postgres-xc-1.2.1 right now. For example, it contains file_fdw, whose create-table-stmt looks like: CREATE FOREIGN T

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-28 Thread David E. Wheeler
On Oct 24, 2014, at 6:36 AM, Alex Goncharov wrote: > Another dimension of the trouble is breaking the operation of the > tools that parse SQL statements for various purposes, e.g. for > dependency analysis. That’s a valid point. > This is a misfeature for the benefit of edit-lazy users only.

Re: [HACKERS] lag_until_you_get_something() OVER () window function

2014-10-28 Thread Merlin Moncure
On Tue, Oct 28, 2014 at 12:40 PM, Kirk Roybal wrote: > Hi Guys, > > I propose a lag (and/or lead) window function that propagates the last > non-null value to the current row. > Here's an example of what I mean by that: > > CREATE TABLE lag_test (id serial primary key, natural_key integer, somebod

[HACKERS] foreign data wrapper option manipulation during Create foreign table time?

2014-10-28 Thread Demai Ni
hi, guys, I am looking for a couple pointers here about fdw, and how to change the option values during CREATE table time. I am using postgres-xc-1.2.1 right now. For example, it contains file_fdw, whose create-table-stmt looks like: CREATE FOREIGN TABLE t1() SERVER file_server OPTIONS(format

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-10-28 Thread Simon Riggs
On 16 October 2014 16:22, Robert Haas wrote: >> Might I gently enquire what the "something usable" we are going to see >> in this release? I'm not up on current plans. > > I don't know how far I'm going to get for this release yet. I think > pg_background is a pretty good milestone, and useful i

Re: [HACKERS] lag_until_you_get_something() OVER () window function

2014-10-28 Thread Vladimir Sitnikov
There is already a patch for that (ignore/respect nulls in lead/lag): https://commitfest.postgresql.org/action/patch_view?id=1096 -- Vladimir -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hack

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 28 October 2014 17:50, Jim Nasby wrote: > On 10/28/14, 9:22 AM, Simon Riggs wrote: >> >> 2. Some additional code in Autovacuum to rebuild corrupt indexes at >> startup, using AV worker processes to perform a REINDEX CONCURRENTLY. > > > I don't think loading more functionality into autovac is th

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 28 October 2014 17:58, Alexander Korotkov wrote: > Also, I'm not sure that many users have enough of courage to use unlogged > AMs. Absence of durability is only half of trouble, another half is lack of > streaming replication. I think if we have unlogged GIN then external > indexing engines w

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 28 October 2014 17:47, Andres Freund wrote: > On 2014-10-28 17:45:36 +, Simon Riggs wrote: >> I'd like to avoid all of the pain by making persistent AMs that are >> recoverable after a crash, rather than during crash recovery. > > Besides the actual difficulities of supporting this, imo not

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Kevin Grittner
Stephen Frost wrote: > the original ask was to be able to view logs as a DBA who isn't a > superuser, and without having to have those views delayed or > complex cron jobs running to set up access to them. I had kinda forgotten it, but I had to set up a cron log rsync at Wisconsin Courts. I und

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Oct 28, 2014 at 11:16 AM, Stephen Frost wrote: > > There are more capabilities that I've been considering longer-term but > > wasn't sure if they should be independent or just lumped into the > > simpler read/write category: > > > > r

Re: [HACKERS] Reducing lock strength of adding foreign keys

2014-10-28 Thread adamrose045
There are actually TWO tables involved: the table upon which the trigger will actually fire, and some other table which is mentioned in passing in the trigger definition. It's possible that the locking requirements for the secondary table are weaker since I don't think the presence of the trig

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Stephen Frost
Kevin, Thanks. * Kevin Grittner (kgri...@ymail.com) wrote: > Stephen my correct me on this, but I seem to remember him saying > that this was part of a general effort to avoid needing to use a > superuser login for routine tasks that don't fit into the area of > what a sysadmin would do. That se

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-28 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/28/2014 05:20 AM, Alvaro Herrera wrote: > Alex Goncharov wrote: > >> This is a misfeature for the benefit of edit-lazy users only. > > +1 +1 Joe - -- Joe Conway credativ LLC: http://www.credativ.us Linux, PostgreSQL, and general Open Source

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-28 Thread Pavel Stehule
2014-10-28 13:20 GMT+01:00 Alvaro Herrera : > Alex Goncharov wrote: > > > This is a misfeature for the benefit of edit-lazy users only. > > +1 > > +1 Pavel > -- > Álvaro Herrerahttp://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training & Services >

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-28 Thread Alvaro Herrera
Alex Goncharov wrote: > This is a misfeature for the benefit of edit-lazy users only. +1 -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-28 Thread Alvaro Herrera
Marti Raudsepp wrote: > On Fri, Oct 24, 2014 at 11:29 AM, Kyotaro HORIGUCHI > wrote: > > - 0001-ALTER-ROLE-CURRENT_USER_v2.patch - the patch. > > +RoleId:CURRENT_USER{ $$ = > "current_user";} > + | USER { $$ = "curr

Re: [HACKERS] [9.4 bug] The database server hangs with write-heavy workload on Windows

2014-10-28 Thread Heikki Linnakangas
On 10/14/2014 03:59 PM, MauMau wrote: BTW, in LWLockWaitForVar(), the first line of the following code fragment is not necessary, because lwWaitLink is set to head immediately. I think it would be good to eliminate as much unnecessary code as possible from the spinlock section. proc->lwWaitL

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Kevin Grittner
Robert Haas wrote: > I think it would help, on all accounts, to explain why in the world > we're spending time on this in the first place. I have a sneaking > suspicion this is 1 of N things we need to do to meet some US > government security standard, and if something like that is the case, > t

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread k...@rice.edu
On Tue, Oct 28, 2014 at 01:51:21PM -0400, Tom Lane wrote: > Simon Riggs writes: > > On 28 October 2014 17:06, Tom Lane wrote: > >> My own thought is that allowing external AMs is simply a natural > >> consequence of PG's general approach to extensibility, and it would > >> be surprising if we wer

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Stephen Frost
* Alexander Korotkov (aekorot...@gmail.com) wrote: > Having access methods as extensions can significantly improves situations > here. Imagine, GIN was an extension. One day we decide to change its binary > format. Then we can issue new extension, GIN2 for instance. User can > upgrade from GIN to G

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Alexander Korotkov
On Tue, Oct 28, 2014 at 8:04 PM, Simon Riggs wrote: > On 28 October 2014 14:22, Simon Riggs wrote: > > > Or put it another way, it will be easier to write new index AMs > > because we'll be able to skip the WAL part until we know we want it. > > To be clear: I am suggesting you do *less* work, n

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Jim Nasby
On 10/28/14, 9:22 AM, Simon Riggs wrote: 2. Some additional code in Autovacuum to rebuild corrupt indexes at startup, using AV worker processes to perform a REINDEX CONCURRENTLY. I don't think loading more functionality into autovac is the right way to do that. -- Jim Nasby, Data Architect, Bl

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Andres Freund
On 2014-10-28 13:37:33 -0400, Tom Lane wrote: > I'm not at all sold on the idea that we need to support dropping AMs. > I think it'd be fine to consider that installing an AM into a given > database is a one-way operation. Then you just need to insert some > pg_depend entries that "pin" the AM's i

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Tom Lane
Simon Riggs writes: > On 28 October 2014 17:06, Tom Lane wrote: >> My own thought is that allowing external AMs is simply a natural >> consequence of PG's general approach to extensibility, and it would >> be surprising if we were to decide we didn't want to allow that. > If it wasn't clear from

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 11:16 AM, Stephen Frost wrote: > There are more capabilities that I've been considering longer-term but > wasn't sure if they should be independent or just lumped into the > simpler read/write category: > > read (eg: importing log files, or importing from an NFS mount) > wr

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 11:33 AM, Adam Brightwell wrote: >> Given that no fewer than four people - all committers - have expressed >> doubts about the design of this patch, I wonder why you're bothering >> to post a new version. > > I understand and my intent was in no way to disregard those conce

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Andres Freund
On 2014-10-28 17:45:36 +, Simon Riggs wrote: > I'd like to avoid all of the pain by making persistent AMs that are > recoverable after a crash, rather than during crash recovery. Besides the actual difficulities of supporting this, imo not being available on HS and directly after a failover es

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 28 October 2014 17:06, Tom Lane wrote: > My own thought is that allowing external AMs is simply a natural > consequence of PG's general approach to extensibility, and it would > be surprising if we were to decide we didn't want to allow that. If it wasn't clear from my two earlier attempts, y

[HACKERS] lag_until_you_get_something() OVER () window function

2014-10-28 Thread Kirk Roybal
Hi Guys, I propose a lag (and/or lead) window function that propagates the last non-null value to the current row. Here's an example of what I mean by that: CREATE TABLE lag_test (id serial primary key, natural_key integer, somebody text); INSERT INTO lag_test(natural_key, somebody) VALUES

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Tom Lane
Andres Freund writes: > On 2014-10-28 13:06:52 -0400, Tom Lane wrote: >> But having said that, it's quite unclear to me that we need the >> CREATE/DROP ACCESS METHOD infrastructure proposed here. The traditional >> theory about that is that if you're competent to develop an AM at all, >> you can

Re: [HACKERS] logical decoding - reading a user catalog table

2014-10-28 Thread Andres Freund
On 2014-10-25 18:18:07 -0400, Steve Singer wrote: > My logical decoding plugin is occasionally getting this error > > "could not resolve cmin/cmax of catalog tuple" > > I get this when my output plugin is trying to read one of the user defined > catalog tables (user_catalog_table=true) Hm. That

Re: [HACKERS] "snapshot too large" error when initializing logical replication (9.4)

2014-10-28 Thread Andres Freund
Hi, On 2014-10-25 18:09:36 -0400, Steve Singer wrote: > I sometimes get the error "snapshot too large" from my logical replication > walsender process when in response to a CREATE_REPLICATION_SLOT. Yes. That's possible if 'too much' was going on until a consistent point was reached. I think we c

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Andres Freund
On 2014-10-28 13:06:52 -0400, Tom Lane wrote: > Stephen Frost writes: > > * Andres Freund (and...@2ndquadrant.com) wrote: > >> The other thing I'm not sure about is that I'm unconvinced that we > >> really want external AMs... > > > I was wondering about this also and curious as to if there's bee

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@2ndquadrant.com) wrote: >> The other thing I'm not sure about is that I'm unconvinced that we >> really want external AMs... > I was wondering about this also and curious as to if there's been any > prior on-list discussion about this proposal that

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 28 October 2014 14:22, Simon Riggs wrote: > Or put it another way, it will be easier to write new index AMs > because we'll be able to skip the WAL part until we know we want it. To be clear: I am suggesting you do *less* work, not more. By allowing AMs to avoid writing WAL we get * higher p

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 28 October 2014 16:19, Stephen Frost wrote: > Would be happy to go back and review earlier discussions, of course, but > I don't recall there being any. It depends how far back you go. I think I've had at least 2 tries at writing something, but not in last 5 years. -- Simon Riggs

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > The other thing I'm not sure about is that I'm unconvinced that we > really want external AMs... I was wondering about this also and curious as to if there's been any prior on-list discussion about this proposal that I've simply missed..? Would be

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Oleg Bartunov
On Tue, Oct 28, 2014 at 7:57 PM, Simon Riggs wrote: > On 28 October 2014 14:53, Robert Haas wrote: > > On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs > wrote: > >> Or put it another way, it will be easier to write new index AMs > >> because we'll be able to skip the WAL part until we know we wan

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-28 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: > Stephen Frost wrote: > > > You can still double-quote reserved words and use them in general. What > > we're talking about here are cases where a word can't be used even if > > it's double-quoted, and we try really hard to keep those cases at an > >

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Andres Freund
Hi, On 2014-10-15 16:08:38 +0400, Alexander Korotkov wrote: > Postgres was initially designed to support access methods extendability. > This extendability lives to present day. However, this is mostly internal > in-core extendability. One can quite easily add new access method into > PostgreSQL c

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 28 October 2014 14:53, Robert Haas wrote: > On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs wrote: >> Or put it another way, it will be easier to write new index AMs >> because we'll be able to skip the WAL part until we know we want it. > > I like the feature you are proposing, but I don't thin

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Adam Brightwell
Robert, > Given that no fewer than four people - all committers - have expressed > doubts about the design of this patch, I wonder why you're bothering > to post a new version. I understand and my intent was in no way to disregard those concerns. The only reason that I have posted a new versio

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-10-28 Thread Rahila Syed
>>>Do we release the buffers for compressed data when fpw is changed from "compress" to "on"? >> The current code does not do this. >Don't we need to do that? Yes this needs to be done in order to avoid memory leak when compression is turned off at runtime while the backend session is running.

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Oct 28, 2014 at 9:24 AM, Stephen Frost wrote: > > That said, it sounds like the primary concern has been if we want this > > feature at all and there hasn't been much discussion of the design > > itself. Comments about the technical design wo

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Andreas Karlsson
On 10/28/2014 04:01 PM, Heikki Linnakangas wrote: Moving on to other issues, isn't 128 bits too small to store the squares of the processed numbers? That could overflow.. Yeah, which is why stddev_*(int8) and var_*(int8) still have to use Numeric in the aggregate state. For the int2 and int4 v

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Heikki Linnakangas
On 10/28/2014 04:47 PM, Andreas Karlsson wrote: On 10/28/2014 03:40 PM, Heikki Linnakangas wrote: The patch doesn't do division with the 128-bit integers. It only does addition and multiplication. Those are pretty straightforward to implement. The patch uses division when converting from __int

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 2:40 AM, Adam Brightwell wrote: > Taking a step back, I'm still not sure I understand the need for this > feature or the use case. It seems to have started as a potential fix to an > inconsistency between ALTER USER and ALTER ROLE syntax (which I think I > could see some v

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 10:22 AM, Simon Riggs wrote: > Or put it another way, it will be easier to write new index AMs > because we'll be able to skip the WAL part until we know we want it. I like the feature you are proposing, but I don't think that we should block Alexander from moving forward

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-28 Thread Kevin Grittner
Stephen Frost wrote: > You can still double-quote reserved words and use them in general. What > we're talking about here are cases where a word can't be used even if > it's double-quoted, and we try really hard to keep those cases at an > absolute minimum. We should also really be keeping a li

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Andreas Karlsson
On 10/28/2014 03:40 PM, Heikki Linnakangas wrote: The patch doesn't do division with the 128-bit integers. It only does addition and multiplication. Those are pretty straightforward to implement. The patch uses division when converting from __int128_t to Numeric. - Andreas -- Sent via pgsq

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Robert Haas
On Tue, Oct 28, 2014 at 9:24 AM, Stephen Frost wrote: > That said, it sounds like the primary concern has been if we want this > feature at all and there hasn't been much discussion of the design > itself. Comments about the technical design would be great. I > appreciate your thoughts about usi

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Heikki Linnakangas
On 10/28/2014 04:06 PM, Tom Lane wrote: Heikki Linnakangas writes: It wouldn't be too hard to just do: struct { int64 high_bits; uint64 low_bits; } pg_int128; and some macros for the + - etc. operators. It might be less work than trying to deal with the portability issues of a

Re: [HACKERS] WIP: Access method extendability

2014-10-28 Thread Simon Riggs
On 15 October 2014 13:08, Alexander Korotkov wrote: > Postgres was initially designed to support access methods extendability. > This extendability lives to present day. However, this is mostly internal > in-core extendability. One can quite easily add new access method into > PostgreSQL core. Bu

Re: [HACKERS] Deferring some AtStart* allocations?

2014-10-28 Thread Andres Freund
On 2014-10-24 11:25:23 -0400, Robert Haas wrote: > On Fri, Oct 24, 2014 at 10:10 AM, Andres Freund > wrote: > > What I was thinking was that you'd append the messages to the layer one > > level deeper than the parent. Then we'd missed the invalidations when > > rolling back the intermediate xact.

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Tom Lane
Heikki Linnakangas writes: > It wouldn't be too hard to just do: > struct { > int64 high_bits; > uint64 low_bits; > } pg_int128; > and some macros for the + - etc. operators. It might be less work than > trying to deal with the portability issues of a native C datatype for this. -1.

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Andres Freund
On 2014-10-28 15:54:30 +0200, Heikki Linnakangas wrote: > On 10/28/2014 03:24 PM, Andres Freund wrote: > >On 2014-10-28 11:05:11 -0200, Arthur Silva wrote: > >>On Sat, Oct 25, 2014 at 12:38 PM, Andreas Karlsson > >>As far as I'm aware int128 types are supported on every major compiler when > >>com

Re: [HACKERS] superuser() shortcuts

2014-10-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > For one I'm less than convinced that the new messages are an > improvement. They seem to be more verbose without a corresponding > improvement in clarity. The goal with the changes is to improve consistency of messaging. These messages are not at

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Heikki Linnakangas
On 10/28/2014 03:24 PM, Andres Freund wrote: On 2014-10-28 11:05:11 -0200, Arthur Silva wrote: On Sat, Oct 25, 2014 at 12:38 PM, Andreas Karlsson As far as I'm aware int128 types are supported on every major compiler when compiling for 64bit platforms. Right? Depends on what you call major. I

Re: [HACKERS] jsonb generator functions

2014-10-28 Thread Andrew Dunstan
On 10/27/2014 05:57 PM, Alvaro Herrera wrote: Andrew Dunstan wrote: This bit: +/* + * Determine how we want to render values of a given type in datum_to_jsonb. + * + * Given the datatype OID, return its JsonbTypeCategory, as well as the type's + * output function OID. If the returned categor

Re: [HACKERS] superuser() shortcuts

2014-10-28 Thread Andres Freund
On 2014-10-28 09:43:35 -0400, Stephen Frost wrote: > All, > > * Stephen Frost (sfr...@snowman.net) wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > > > As I started looking at this, there are multiple other places where > > > > these types of error messages occur (opclasscmds.c, u

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-28 Thread Heikki Linnakangas
On 10/27/2014 06:12 PM, Heikki Linnakangas wrote: On 10/27/2014 02:12 PM, Fujii Masao wrote: >On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas > wrote: >>On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: >>> >>>At least for master, we should consider changing the way the archiving >>>work

Re: [HACKERS] superuser() shortcuts

2014-10-28 Thread Stephen Frost
All, * Stephen Frost (sfr...@snowman.net) wrote: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > > As I started looking at this, there are multiple other places where > > > these types of error messages occur (opclasscmds.c, user.c, > > > postinit.c, miscinit.c are just a few), not just a

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-10-28 09:24:18 -0400, Stephen Frost wrote: > > There is no doubt that consensus on the desirability and design needs > > to be reached before we can even consider committing it. I suspect > > Adam posted it simply because he had identified

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Andreas Karlsson
On 10/28/2014 02:05 PM, Arthur Silva wrote: As far as I'm aware int128 types are supported on every major compiler when compiling for 64bit platforms. Right? Both gcc and clang support __int128_t, but I do not know about other compilers like icc and MSVC. Andreas -- Sent via pgsql-hackers

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Andres Freund
On 2014-10-28 09:24:18 -0400, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > There is absolutely NOT consensus on > > this design or anything close to it. > > There is no doubt that consensus on the desirability and design needs > to be reached before we can even consider

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Andres Freund
On 2014-10-28 11:05:11 -0200, Arthur Silva wrote: > On Sat, Oct 25, 2014 at 12:38 PM, Andreas Karlsson > As far as I'm aware int128 types are supported on every major compiler when > compiling for 64bit platforms. Right? Depends on what you call major. IIRC some not that old msvc versions don't f

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > There is absolutely NOT consensus on > this design or anything close to it. There is no doubt that consensus on the desirability and design needs to be reached before we can even consider committing it. I suspect Adam posted it simply because he had

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Merlin Moncure
On Sat, Oct 25, 2014 at 9:38 AM, Andreas Karlsson wrote: > Hi, > > There was recently talk about if we should start using 128-bit integers > (where available) to speed up the aggregate functions over integers which > uses numeric for their internal state. So I hacked together a patch for this > to

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-28 Thread Robert Haas
On Mon, Oct 27, 2014 at 5:59 PM, Adam Brightwell wrote: > Attached is a patch with minor updates/corrections. Given that no fewer than four people - all committers - have expressed doubts about the design of this patch, I wonder why you're bothering to post a new version. It seems to me that you

Re: [HACKERS] [WIP Patch] Using 128-bit integers for sum, avg and statistics aggregates

2014-10-28 Thread Arthur Silva
On Sat, Oct 25, 2014 at 12:38 PM, Andreas Karlsson wrote: > Hi, > > There was recently talk about if we should start using 128-bit integers > (where available) to speed up the aggregate functions over integers which > uses numeric for their internal state. So I hacked together a patch for > this

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-28 Thread Stephen Frost
* Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: > > +RoleId:CURRENT_USER{ $$ = > > "current_user";} > > + | USER { $$ = "current_user";} > > + | CURRENT_ROLE { $$ =

  1   2   >