Re: [HACKERS] Something flaky in the "relfilenode mapping" infrastructure

2014-06-11 Thread Tom Lane
Andres Freund writes: > On 2014-06-12 00:38:36 -0400, Noah Misch wrote: >> http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog&dt=2014-06-12%2000%3A17%3A07 > Hm. My guess it's that it's just a 'harmless' concurrency issue. The > test currently run in concurrency with others: I think

Re: [HACKERS] Something flaky in the "relfilenode mapping" infrastructure

2014-06-11 Thread Andres Freund
On 2014-06-12 00:38:36 -0400, Noah Misch wrote: > On Tue, Apr 15, 2014 at 03:28:41PM -0400, Tom Lane wrote: > > Andres Freund writes: > > >> On 2014-03-27 08:02:35 -0400, Tom Lane wrote: > > >>> Buildfarm member prairiedog thinks there's something unreliable about > > >>> commit f01d1ae3a104019d6d

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Pavel Stehule
2014-06-12 7:08 GMT+02:00 Tom Lane : > Noah Misch writes: > > On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: > >> Meanwhile, we have to either revert the addition of lo_create(oid, > >> bytea) altogether, or choose a different name for it. Suggestions? > > > lo_new() or lo_make()? An

Re: [HACKERS] Something flaky in the "relfilenode mapping" infrastructure

2014-06-11 Thread Tom Lane
Noah Misch writes: > The improved test just tripped: > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog&dt=2014-06-12%2000%3A17%3A07 Ugh. If the MTBF is circa three months, how will we catch this before we're dead? A quick look around the machine's buildfarm directory says ther

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Pavel Stehule
2014-06-12 6:54 GMT+02:00 Noah Misch : > On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: > > While there's certainly a good argument to be made for making > > lo_initialize do that query differently, there's no way that we > > can fix every copy of libpq that's in the field. I think we

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Tom Lane
Noah Misch writes: > On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: >> Meanwhile, we have to either revert the addition of lo_create(oid, >> bytea) altogether, or choose a different name for it. Suggestions? > lo_new() or lo_make()? An earlier draft of the patch that added > lo_creat

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Pavel Stehule
2014-06-12 6:22 GMT+02:00 Tatsuo Ishii : > > Meanwhile, we have to either revert the addition of lo_create(oid, > > bytea) altogether, or choose a different name for it. Suggestions? > > I wonder if there's any use case where we need to store bytea into > large objects. Don't we already have byte

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Noah Misch
On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: > While there's certainly a good argument to be made for making > lo_initialize do that query differently, there's no way that we > can fix every copy of libpq that's in the field. I think we have to > consider that "there can be only one l

Re: [HACKERS] Something flaky in the "relfilenode mapping" infrastructure

2014-06-11 Thread Noah Misch
On Tue, Apr 15, 2014 at 03:28:41PM -0400, Tom Lane wrote: > Andres Freund writes: > >> On 2014-03-27 08:02:35 -0400, Tom Lane wrote: > >>> Buildfarm member prairiedog thinks there's something unreliable about > >>> commit f01d1ae3a104019d6d68aeff85c4816a275130b3: > > > So I had made a notice to r

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Tatsuo Ishii
> Meanwhile, we have to either revert the addition of lo_create(oid, > bytea) altogether, or choose a different name for it. Suggestions? I wonder if there's any use case where we need to store bytea into large objects. Don't we already have bytea data type? If the use case is for large data whic

Re: [HACKERS] Proposing pg_hibernate

2014-06-11 Thread Gurjeet Singh
On Wed, Jun 11, 2014 at 10:56 AM, Robert Haas wrote: > On Tue, Jun 10, 2014 at 10:03 PM, Gurjeet Singh wrote: >> And it's probably accepted by now that such a bahviour is not >> catastrophic, merely inconvenient. > > I think the whole argument for having pg_hibernator is that getting > the block

Re: [HACKERS] Proposing pg_hibernate

2014-06-11 Thread Gurjeet Singh
On Wed, Jun 11, 2014 at 12:25 AM, Amit Kapila wrote: > On Wed, Jun 11, 2014 at 7:59 AM, Gurjeet Singh wrote: >> On Sun, Jun 8, 2014 at 3:24 AM, Amit Kapila >> wrote: >> > Yeap, but if it crashes before writing checkpoint record, it will lead >> > to >> > recovery which is what we are considering

[HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Tom Lane
While investigating a different issue, I was astonished to find that pg_restore in HEAD is incapable of restoring dumps containing large objects: it fails with messages like pg_restore: [archiver] could not create large object 10: ERROR: function call message contains 1 arguments but functio

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-06-11 Thread Amit Kapila
On Tue, May 27, 2014 at 10:35 AM, Amit Kapila wrote: > On Sun, May 11, 2014 at 11:23 PM, Tom Lane wrote: > > I think it's clearly *necessary* to forbid setting data_directory in > > postgresql.auto.conf. The file is defined to be found in the data > > directory, so any such setting is circular l

[HACKERS] Few observations in replication slots related code

2014-06-11 Thread Amit Kapila
Few observations in Replication slots related code: 1. In function StartupReplicationSlots(XLogRecPtr checkPointRedo), parameter checkPointRedo is not used. 2. Few check are in different order in functions pg_create_physical_replication_slot() and pg_create_logical_replication_slot(). if (get_c

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Robert Haas
On Wed, Jun 11, 2014 at 8:59 PM, Stephen Frost wrote: >> Row-level security is not a chance for the system to deny access; it's >> a chance for user-defined code to take control and perform arbitrary >> operations. So the scope of what we're contemplating for row-level >> security is really far,

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Tom Lane
Noah Misch writes: > Based on the commit message and procedural history, I thought commit 6513633 > was changing behavior solely for the combination of "\pset expanded" and > "\pset format wrapped". Peter's and my test cases show that it also changed > behavior for "\pset expanded" alone. That's

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Noah Misch
On Wed, Jun 11, 2014 at 08:59:34PM +0100, Greg Stark wrote: > The leading space that you (ie Peter) are complaining about in: > > col | 1 > +-- > col | 2 > > Is there because if the cell wrapped it would get an ellipsis (ie > '...' but it's a single unicode character) in that column to indi

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jun 11, 2014 at 12:23 PM, Stephen Frost wrote: > > This argument could have been made for column-level privileges also, no? > > Not really. First of all, we didn't have security_barrier views at > that time, let alone security barrier views

Re: [HACKERS] [patch] build issues on Win32

2014-06-11 Thread Noah Misch
On Wed, Mar 10, 2010 at 10:51:23AM -0500, Tom Lane wrote: > =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= writes: > > -LDFLAGS="-Wl,--allow-multiple-definition" > > +LDFLAGS="${LDFLAGS} -Wl,--allow-multiple-definition" > > That bit seems sane. I've committed that part. Thanks. -- Noah Misch Enterprise

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> I'm really concerned about the security implications of this patch. I > >> think we're setting ourselves up for a whole lot of hurt for somewhat > >> unclear gain. > > > I'm

Re: [HACKERS] [GENERAL] Question about partial functional indexes and the query planner

2014-06-11 Thread Tom Lane
Robert Haas writes: > On Tue, Jun 10, 2014 at 7:19 PM, Tom Lane wrote: >> Given the lack of previous complaints, I'm not sure this amounts to >> a back-patchable bug, but it does seem like something worth fixing >> going forward. > Agreed, although I'd be willing to see us slip it into 9.4. It'

Re: [HACKERS] view reloptions

2014-06-11 Thread Fabrízio de Royes Mello
On Wed, Jun 11, 2014 at 4:46 PM, Alvaro Herrera wrote: > I just noticed by chance that view relations are using StdRdOptions to > allocate their reloptions. I can't find any reason for this, other than > someone failed to realize that they should instead have a struct defined > of its own, just

Re: [HACKERS] view reloptions

2014-06-11 Thread Tom Lane
Alvaro Herrera writes: > Is it too late to redefine the check_option_offset stuff before 9.4 > ships, so that it is in its own struct? We have a forced initdb already for beta2, so I'd say not. > 3) I don't have time to do the legwork before CF1 anyway ... but if nobody does the work, it's moot

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Greg Stark
And Gmail has thoroughly mangled that email. Let me see if I can resend it from Emacs more clearly. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Greg Stark
On Wed, Jun 11, 2014 at 7:52 PM, Peter Eisentraut wrote: > On 6/8/14, 11:29 PM, Noah Misch wrote: >> The patch did not restore 9.3 behavior for that one. Starting with commit >> 6513633, the first line of letters is space-padded on the right to the width >> of the second line of letters. To illu

[HACKERS] view reloptions

2014-06-11 Thread Alvaro Herrera
I just noticed by chance that view relations are using StdRdOptions to allocate their reloptions. I can't find any reason for this, other than someone failed to realize that they should instead have a struct defined of its own, just like (say) GIN indexes do. Views using StdRdOptions is quite poi

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Robert Haas
On Wed, Jun 11, 2014 at 12:23 PM, Stephen Frost wrote: >> In my view, commit 842faa714c0454d67e523f5a0b6df6500e9bc1a5 basically >> *is* row-level security: instead of applying a row-level security >> policy to a table, just create a security-barrier view over the table >> and grant access to the v

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Peter Eisentraut
On 6/8/14, 11:29 PM, Noah Misch wrote: > The patch did not restore 9.3 behavior for that one. Starting with commit > 6513633, the first line of letters is space-padded on the right to the width > of the second line of letters. To illustrate, I have attached raw psql output > from both commit 6513

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Tom Lane
Stephen Frost writes: > * Robert Haas (robertmh...@gmail.com) wrote: >> I'm really concerned about the security implications of this patch. I >> think we're setting ourselves up for a whole lot of hurt for somewhat >> unclear gain. > I'm certainly of a different opinion and, for the most part, I

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Greg Stark
On Wed, Jun 11, 2014 at 3:26 PM, Tom Lane wrote: > If we didn't have mechanisms like this, we'd have far worse hazards from > ALTER TABLE than whether the planner made an incorrect join optimization. > Consider ALTER COLUMN TYPE for instance. Obviously not general cases of ALTER COLUMN TYPE but d

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > I'm really concerned about the security implications of this patch. I > think we're setting ourselves up for a whole lot of hurt for somewhat > unclear gain. I'm certainly of a different opinion and, for the most part, I feel that if there are securi

[HACKERS] [GSoC2014] Patch ALTER TABLE ... SET LOGGED

2014-06-11 Thread Fabrízio de Royes Mello
Hi all, As part of GSoC2014 I'm sending a patch to add the capability of change an unlogged table to logged [1]. I'll add it to the 9.5CF1. Regards, [1] https://wiki.postgresql.org/wiki/Allow_an_unlogged_table_to_be_changed_to_logged_GSoC_2014 -- Fabrízio de Royes Mello Consultoria/Coaching P

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 06/11/2014 07:24 AM, Tom Lane wrote: > > Is the point of that that the table owner might have put trojan-horse > > functions into the RLS qual? If so, why are we only concerned about > > defending the superuser and not other users? Seems like the

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 06/11/2014 02:19 AM, Tom Lane wrote: > > Hm ... I'm not following why we'd need a special case for superusers and > > not anyone else? Seems like any useful RLS scheme is going to require > > more privilege levels than just superuser and not-super

Re: [HACKERS] Proposing pg_hibernate

2014-06-11 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:03 PM, Gurjeet Singh wrote: > And it's probably accepted by now that such a bahviour is not > catastrophic, merely inconvenient. I think the whole argument for having pg_hibernator is that getting the block cache properly initialized is important. If it's not important

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Tom Lane
Andres Freund writes: > Your wish just seems like a separate feature to me. Including > replication commands in 'all' seems correct independent of the desire > for a more granular control. No, I think I've got to vote with the other side on that. The reason we can have log_statement as a scalar

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Robert Haas
On Tue, Jun 10, 2014 at 7:18 PM, Craig Ringer wrote: > On 06/11/2014 02:19 AM, Tom Lane wrote: >> Hm ... I'm not following why we'd need a special case for superusers and >> not anyone else? Seems like any useful RLS scheme is going to require >> more privilege levels than just superuser and not-

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Tom Lane
Marti Raudsepp writes: > On Wed, Jun 11, 2014 at 11:53 AM, David Rowley wrote: >> as long as the transaction id >> is taken before we start planning in session 1 then it should not matter if >> another session drops the constraint and inserts a NULL value as we won't >> see that NULL value in our

Re: [HACKERS] [GENERAL] Question about partial functional indexes and the query planner

2014-06-11 Thread Robert Haas
On Tue, Jun 10, 2014 at 7:19 PM, Tom Lane wrote: > Given the lack of previous complaints, I'm not sure this amounts to > a back-patchable bug, but it does seem like something worth fixing > going forward. Agreed, although I'd be willing to see us slip it into 9.4. It's doubtful that anyone will

Re: [HACKERS] Inaccuracy in VACUUM's tuple count estimates

2014-06-11 Thread Kevin Grittner
Andres Freund wrote: > On 2014-06-09 11:24:22 -0700, Kevin Grittner wrote: >> The only way that it could be a problem is if the DELETE is in a >> subtransaction which might get rolled back without rolling back the >> INSERT. > > The way I understand the code in that case the subxid in xmax would

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-11 Thread Fujii Masao
On Tue, Jun 10, 2014 at 5:01 PM, wrote: >> No. IIUC walreceiver does flush *less* frequently than what you >> implemented on pg_receivexlog. Your version of pg_receivexlog tries to >> do flush every time when it receives one WAL chunk. OTOH, walreceiver >> does flush only when there is no extra W

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 14:50:34 +0200, Magnus Hagander wrote: > On Wed, Jun 11, 2014 at 2:35 PM, Andres Freund > wrote: > > > On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: > > > Yes, but how would you specify for example "i want DDL and all > > replication > > > commands" (which is quite a reasonab

Re: [HACKERS] Inaccuracy in VACUUM's tuple count estimates

2014-06-11 Thread Andres Freund
On 2014-06-09 11:24:22 -0700, Kevin Grittner wrote: > Andres Freund wrote: > > On 2014-06-09 09:45:12 -0700, Kevin Grittner wrote: > > > I am not sure, given predicate.c's coding, how > > HEAPTUPLE_DELETE_IN_PROGRESS could cause problems. Could you elaborate, > > since that's the contentious poin

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 2:35 PM, Andres Freund wrote: > On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: > > Yes, but how would you specify for example "i want DDL and all > replication > > commands" (which is quite a reasonable thing to log, I believe). If you > > actually require it to be s

Re: [HACKERS] tests for client programs

2014-06-11 Thread Andres Freund
Hi, On 2014-06-04 20:40:40 -0400, Peter Eisentraut wrote: > On Wed, 2014-05-07 at 03:08 +0200, Andres Freund wrote: > > > As an additional issue it currently doesn't seem to work in VPATH > > > builds. That's imo a must fix. > > > > A "cd $(srcdir) && .." in prove_installcheck and prove_check see

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: > Yes, but how would you specify for example "i want DDL and all replication > commands" (which is quite a reasonable thing to log, I believe). If you > actually require it to be set to "all", most people won't have any use at > all for it... Th

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 2:17 PM, Andres Freund wrote: > On 2014-06-11 13:42:58 +0200, Magnus Hagander wrote: > > On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao > wrote: > > > > > Hi, > > > > > > Replication commands like IDENTIFY_COMMAND are not logged even when > > > log_statements is set to all

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 13:42:58 +0200, Magnus Hagander wrote: > On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao wrote: > > > Hi, > > > > Replication commands like IDENTIFY_COMMAND are not logged even when > > log_statements is set to all. Some users who use log_statements to > > audit *all* statements might

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao wrote: > Hi, > > Replication commands like IDENTIFY_COMMAND are not logged even when > log_statements is set to all. Some users who use log_statements to > audit *all* statements might dislike this current situation. So I'm > thinking to change log_st

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 19:34:25 +0900, Fujii Masao wrote: > Hi, > > Replication commands like IDENTIFY_COMMAND are not logged even when > log_statements is set to all. Some users who use log_statements to > audit *all* statements might dislike this current situation. So I'm > thinking to change log_stateme

Re: [HACKERS] Compression of full-page-writes

2014-06-11 Thread Pavan Deolasee
On Wed, Jun 11, 2014 at 4:19 PM, Fujii Masao wrote: > > IIUC even when we adopt only one algorithm, additional at least one bit is > necessary to see whether this backup block is compressed or not. > > This flag is necessary only for backup block, so there is no need to use > the header of each W

Re: [HACKERS] Compression of full-page-writes

2014-06-11 Thread Fujii Masao
On Wed, Jun 11, 2014 at 10:05 AM, Michael Paquier wrote: > On Tue, Jun 10, 2014 at 11:49 PM, Rahila Syed wrote: >> Hello , >> >> >> In order to facilitate changing of compression algorithms and to be able to >> recover using WAL records compressed with different compression algorithms, >> inform

[HACKERS] replication commands and log_statements

2014-06-11 Thread Fujii Masao
Hi, Replication commands like IDENTIFY_COMMAND are not logged even when log_statements is set to all. Some users who use log_statements to audit *all* statements might dislike this current situation. So I'm thinking to change log_statements or add something like log_replication so that we can log

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Wed, Jun 11, 2014 at 9:32 PM, Marti Raudsepp wrote: > On Sun, Jun 8, 2014 at 3:36 PM, David Rowley wrote: > > Currently pull_up_sublinks_qual_recurse only changes the plan for NOT > EXISTS > > queries and leaves NOT IN alone. The reason for this is because the > values > > returned by a subqu

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Marti Raudsepp
On Sun, Jun 8, 2014 at 3:36 PM, David Rowley wrote: > Currently pull_up_sublinks_qual_recurse only changes the plan for NOT EXISTS > queries and leaves NOT IN alone. The reason for this is because the values > returned by a subquery in the IN clause could have NULLs. There's a bug in targetListIs

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Marti Raudsepp
On Wed, Jun 11, 2014 at 11:53 AM, David Rowley wrote: >> The only way to consistently guarantee nullability is through primary >> key constraints. Fortunately that addresses most of the use cases of >> NOT IN(), in my experience. >> See the comment in check_functional_grouping: > I saw that, but

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Tue, Jun 10, 2014 at 2:19 PM, Tom Lane wrote: > Jeff Janes writes: > > If you are using NOT IN, then once you find a NULL in the outer input (if > > the outer input is the in-list: clearly you can't reverse the two inputs > in > > this case), you don't even need to finish reading the outer in

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Mon, Jun 9, 2014 at 11:20 PM, Marti Raudsepp wrote: > On Sun, Jun 8, 2014 at 3:36 PM, David Rowley wrote: > > Currently pull_up_sublinks_qual_recurse only changes the plan for NOT > EXISTS > > queries and leaves NOT IN alone. The reason for this is because the > values > > returned by a subqu