Re: [HACKERS] Re: [ADMIN] problems using pg_start_backup/pg_stop_backup and pg_basebackup at same time

2016-07-01 Thread Amit Kapila
On Fri, Jul 1, 2016 at 5:00 PM, Magnus Hagander wrote: > > > On Mon, Jun 27, 2016 at 8:27 PM, Alex Malek wrote: >> >> Should I file a bug? > > > Oh, sorry. I did not spot that this was posted to pgsql-admin, which has a > lot fewer readers. I've moved it over to -hackers where more people will se

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
On Fri, Jul 1, 2016 at 10:01 PM, Noah Misch wrote: >> I'm happy that I can do at least that much, but I see no reason to not >> go significantly further. > > Don't risk bundling tests for other sorting scenarios. A minimal test for the > bug in question helps to qualify your patch as an exemplary

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Noah Misch
On Fri, Jul 01, 2016 at 09:12:42PM -0700, Peter Geoghegan wrote: > On Fri, Jul 1, 2016 at 8:37 PM, Noah Misch wrote: > > PostgreSQL is open to moving features from zero test coverage to nonzero > > test > > coverage. The last several releases have each done so. Does that > > sufficiently clarif

Re: [HACKERS] Forthcoming SQL standards about JSON and Multi-Dimensional Arrays (FYI)

2016-07-01 Thread Pavel Stehule
Hi 2016-07-01 20:46 GMT+02:00 Peter Eisentraut < peter.eisentr...@2ndquadrant.com>: > On 7/1/16 7:06 AM, Craig Ringer wrote: > >> Yeah, but since when has the SQL standard adopted any existing >> implementation's spelling of a feature? It seems to be politically >> impossible. >> > > The SQL/JSON

Re: [HACKERS] fixing subplan/subquery confusion

2016-07-01 Thread Amit Kapila
On Sat, Jul 2, 2016 at 12:29 AM, Robert Haas wrote: > On Thu, Jun 30, 2016 at 6:49 PM, Robert Haas wrote: >> I haven't had a chance to do this yet, so I'm going to do it tomorrow >> instead. > > I dug into this a bit and found more problems. I wondered why Tom's > patch did this: > > !

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
On Fri, Jul 1, 2016 at 8:37 PM, Noah Misch wrote: > Yes, please produce a patch version bearing a regression test that exercises > the bug. I feel it counts even if the you need Valgrind Memcheck to see that > it exercises the bug, but I won't interfere if Robert disagrees. The test > should tak

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Noah Misch
On Fri, Jul 01, 2016 at 08:09:07PM -0700, Peter Geoghegan wrote: > On Fri, Jul 1, 2016 at 7:05 PM, Noah Misch wrote: > > I think such a test would suffice to cover this bug if Valgrind Memcheck > > does > > detect the problem in it. > > Are you asking me to produce a regression test that exercis

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Amit Kapila
On Fri, Jul 1, 2016 at 7:52 PM, Masahiko Sawada wrote: > On Fri, Jul 1, 2016 at 11:12 AM, Amit Kapila wrote: >> >> Why do you think IndexOnlyScan will return wrong result? If the >> server crash in the way as you described, the transaction that has >> made modifications will anyway be considered

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Amit Kapila
On Sat, Jul 2, 2016 at 12:53 AM, Andres Freund wrote: > On 2016-07-01 15:18:39 -0400, Robert Haas wrote: >> On Fri, Jul 1, 2016 at 10:22 AM, Masahiko Sawada >> wrote: >> > Ah, you're right, I misunderstood. >> > >> > Attached updated patch incorporating your comments. >> > I've changed it so tha

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
On Fri, Jul 1, 2016 at 7:05 PM, Noah Misch wrote: > I think such a test would suffice to cover this bug if Valgrind Memcheck does > detect the problem in it. Are you asking me to produce a regression test that exercises the bug? I would be happy to do so, but I require some clarification on proj

Re: [HACKERS] Forthcoming SQL standards about JSON and Multi-Dimensional Arrays (FYI)

2016-07-01 Thread Andreas Karlsson
On 07/01/2016 07:31 PM, Andres Freund wrote: Yeah, but since when has the SQL standard adopted any existing implementation's spelling of a feature? It seems to be politically impossible. Especially if nobody really lobbies on part of postgresql. Has any of the major PostgreSQL companies looke

Re: [HACKERS] Documentation fixes for pg_visibility

2016-07-01 Thread Amit Kapila
On Fri, Jul 1, 2016 at 9:30 PM, Robert Haas wrote: > On Fri, Jul 1, 2016 at 10:09 AM, Amit Kapila wrote: Ah, I see. So your suggestion is to do this job in lazy_scan_heap() when scanning each block, and then to issue a WARNING and clear the visibility map. Indeed that's better. I g

Re: [HACKERS] Comment and function argument names are mismatched in bugmgr.c.

2016-07-01 Thread Masahiko Sawada
On Sat, Jul 2, 2016 at 6:34 AM, Andres Freund wrote: > Hi, > > On 2016-06-23 16:24:12 +0900, Masahiko Sawada wrote: >> By commit 428b1d6b, function WritebackContextInit is added but the >> comment of this function seems to be incorrect. >> *max_coalesce variable doesn't exist at anywhere. >> Also,

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Noah Misch
On Fri, Jul 01, 2016 at 12:37:23PM -0700, Peter Geoghegan wrote: > On Fri, Jul 1, 2016 at 12:30 PM, Peter Geoghegan wrote: > > Checkout my gensort tool from github. Build the C tool with "make". > > Then, from the working directory: > > > > ./postgres_load.py -m 250 --skew --logged > > psql -c "CR

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Andreas Karlsson
On 07/02/2016 02:45 AM, Andreas Karlsson wrote: On 07/02/2016 02:28 AM, Alvaro Herrera wrote: Generally, version number tests sprinkled all over the place are not terribly nice. I think it would be better to get configure to define a symbol like HAVE_BIO_METH_NEW. Not sure about the other hunk

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Andreas Karlsson
On 07/02/2016 02:28 AM, Alvaro Herrera wrote: static BIO_METHOD * my_BIO_s_socket(void) { - if (!my_bio_initialized) + if (!my_bio_methods) { - memcpy(&my_bio_methods, BIO_s_socket(), sizeof(BIO_METHOD)); - my_bio_methods.bread = my_sock_read; -

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Alvaro Herrera
Thanks for this effort. > static BIO_METHOD * > my_BIO_s_socket(void) > { > - if (!my_bio_initialized) > + if (!my_bio_methods) > { > - memcpy(&my_bio_methods, BIO_s_socket(), sizeof(BIO_METHOD)); > - my_bio_methods.bread = my_sock_read; > - my_

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Andreas Karlsson
On 07/01/2016 11:41 AM, Christoph Berg wrote: thanks for the patches. I applied all there patches on top of HEAD (10c0558f). The server builds and passes "make check", pgcrypto still needs work, though: Thanks, I had forgotten pgcrypto. When fixing pgcrypto I noticed that the OpenSSL team has

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-07-01 Thread Karl O. Pinc
On Tue, 28 Jun 2016 11:06:24 +0200 Gilles Darold wrote: > Thank you very much for the patch review and please apologies this too > long response delay. I was traveling since end of April and totally > forgotten this patch. I have applied all your useful feedbacks on the > patch and attached a new

Re: [HACKERS] [sqlsmith] ERROR: plan should not reference subplan's variable

2016-07-01 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 1, 2016 at 5:29 PM, Tom Lane wrote: >> ... but if I try to actually execute the query, it crashes at runtime, >> apparently because the CTE has not been passed over to the parallel >> worker. Robert, is it expected that CTEs should be parallel-safe? >> I'd have

Re: [HACKERS] [sqlsmith] ERROR: plan should not reference subplan's variable

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 5:29 PM, Tom Lane wrote: > Andreas Seltenreich writes: >> Updating master from f8c5855..1bdae16, sqlsmith triggers "failed to >> generate plan" errors again. Below is the smallest query logged so far. > > Hmm, interesting. This can be reduced to > > set force_parallel_mod

Re: [HACKERS] Broken handling of lwlocknames.h

2016-07-01 Thread Michael Paquier
On Sat, Jul 2, 2016 at 4:13 AM, Tom Lane wrote: > Michael Paquier writes: >> Yes that would be indeed cleaner this way. I have poked a bit at that >> and finished with the attached that defines some rules to generate all >> the files needed. > > I made some mostly-cosmetic changes to this and pus

Re: [HACKERS] pg_replication_origin_xact_reset() and its argument variables

2016-07-01 Thread Tom Lane
Andres Freund writes: > On 2016-06-30 10:14:04 -0400, Tom Lane wrote: >> Fujii Masao writes: >>> As far as I read the code of the function, those arguments don't seem to >>> be necessary. So I'm afraid that the pg_proc entry for the function might >>> be incorrect and those two arguments should b

Re: [HACKERS] pg_replication_origin_xact_reset() and its argument variables

2016-07-01 Thread Andres Freund
On 2016-06-30 10:14:04 -0400, Tom Lane wrote: > Fujii Masao writes: > > The document explains that pg_replication_origin_xact_reset() doesn't have > > any argument variables. But it's been actually defined so as to have two > > argument variables with pg_lsn and timestamptz data types, as follows.

Re: [HACKERS] Comment and function argument names are mismatched in bugmgr.c.

2016-07-01 Thread Andres Freund
Hi, On 2016-06-23 16:24:12 +0900, Masahiko Sawada wrote: > By commit 428b1d6b, function WritebackContextInit is added but the > comment of this function seems to be incorrect. > *max_coalesce variable doesn't exist at anywhere. > Also, I think it should be fixed that the argument name of this > fu

Re: [HACKERS] [sqlsmith] ERROR: plan should not reference subplan's variable

2016-07-01 Thread Tom Lane
Andreas Seltenreich writes: > Updating master from f8c5855..1bdae16, sqlsmith triggers "failed to > generate plan" errors again. Below is the smallest query logged so far. Hmm, interesting. This can be reduced to set force_parallel_mode = on; explain with j1 as (select * from int8_tbl) select

Re: [HACKERS] _mdfd_getseg can be expensive

2016-07-01 Thread Peter Geoghegan
On Thu, Jun 30, 2016 at 7:08 PM, Andres Freund wrote: > If you have a big enough index (maybe ~150GB+), sure. Before that, > probably not. > > It's usually pretty easy to see in cpu profiles whether this issue > exists. I think that this is a contributing factor to why merging in parallel CREATE

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Jim Nasby
On 7/1/16 3:43 PM, Andres Freund wrote: On 2016-07-01 15:42:22 -0500, Jim Nasby wrote: On 7/1/16 2:23 PM, Andres Freund wrote: The only cost of that is that vacuum will come along and mark the page all-visible again instead of skipping it, but that's probably not an enormous expense in most cas

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Andres Freund
On 2016-07-01 15:42:22 -0500, Jim Nasby wrote: > On 7/1/16 2:23 PM, Andres Freund wrote: > > > > The only > > > > cost of that is that vacuum will come along and mark the page > > > > all-visible again instead of skipping it, but that's probably not an > > > > enormous expense in most cases. > > I

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Jim Nasby
On 7/1/16 2:23 PM, Andres Freund wrote: > The only > cost of that is that vacuum will come along and mark the page > all-visible again instead of skipping it, but that's probably not an > enormous expense in most cases. I think the main cost is not having the page marked as all-visible for index

Re: [HACKERS] Bad behavior from plpython 'return []'

2016-07-01 Thread Jim Nasby
On 7/1/16 2:52 PM, Tom Lane wrote: + /* if caller tries to specify zero-length array, make it empty */ + if (nelems <= 0) + return construct_empty_array(elmtype); + /* compute required space */ nbytes = 0; hasnulls = false; But that might introdu

Re: [HACKERS] Bad behavior from plpython 'return []'

2016-07-01 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 30, 2016 at 9:25 PM, Jim Nasby wrote: >> SELECT array_dims(pg_temp.bad()), array_dims('{}'::text[]); >> array_dims | array_dims >> + >> [1:0] | >> (1 row) > Yeah, that's a bug. It looks like this is because PLySequence_ToArray negle

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
On Fri, Jul 1, 2016 at 12:30 PM, Peter Geoghegan wrote: > Checkout my gensort tool from github. Build the C tool with "make". > Then, from the working directory: > > ./postgres_load.py -m 250 --skew --logged > psql -c "CREATE INDEX segfaults on sort_test_skew(sortkey);" > psql -c "CLUSTER sort_tes

[HACKERS] [sqlsmith] ERROR: plan should not reference subplan's variable

2016-07-01 Thread Andreas Seltenreich
Updating master from f8c5855..1bdae16, sqlsmith triggers "failed to generate plan" errors again. Below is the smallest query logged so far. regards, Andreas -- ERROR: plan should not reference subplan's variable set force_parallel_mode = 'on'; set max_parallel_workers_per_gather = '1'; explain

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
On Fri, Jul 1, 2016 at 12:10 PM, Robert Haas wrote: >> I could give you steps to reproduce the bug, but they involve creating >> a large table using my gensort tool [1]. It isn't trivial. Are you >> interested? > > The bug can't very well be so simple that you need not include a set > of steps to

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Andres Freund
On 2016-07-01 15:18:39 -0400, Robert Haas wrote: > On Fri, Jul 1, 2016 at 10:22 AM, Masahiko Sawada > wrote: > > Ah, you're right, I misunderstood. > > > > Attached updated patch incorporating your comments. > > I've changed it so that heap_xlog_lock clears vm flags if page is > > marked all froz

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 10:22 AM, Masahiko Sawada wrote: > Ah, you're right, I misunderstood. > > Attached updated patch incorporating your comments. > I've changed it so that heap_xlog_lock clears vm flags if page is > marked all frozen. I believe that this should be separated into two patches, s

Re: [HACKERS] Broken handling of lwlocknames.h

2016-07-01 Thread Tom Lane
Michael Paquier writes: > Yes that would be indeed cleaner this way. I have poked a bit at that > and finished with the attached that defines some rules to generate all > the files needed. I made some mostly-cosmetic changes to this and pushed it. One thing to note is that it seemed to me you'd

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 2:23 PM, Peter Geoghegan wrote: > On Fri, Jul 1, 2016 at 6:23 AM, Robert Haas wrote: >> The proposed patch contains no test case and no description of how to >> reproduce the problem. I am not very keen on the idea of trying to >> puzzle that out from first principles. > >

Re: [HACKERS] fixing subplan/subquery confusion

2016-07-01 Thread Robert Haas
On Thu, Jun 30, 2016 at 6:49 PM, Robert Haas wrote: > I haven't had a chance to do this yet, so I'm going to do it tomorrow instead. I dug into this a bit and found more problems. I wondered why Tom's patch did this: ! if (has_parallel_hazard((Node *) rte->subquery, false)

Re: [HACKERS] Forthcoming SQL standards about JSON and Multi-Dimensional Arrays (FYI)

2016-07-01 Thread Peter Eisentraut
On 7/1/16 7:06 AM, Craig Ringer wrote: Yeah, but since when has the SQL standard adopted any existing implementation's spelling of a feature? It seems to be politically impossible. The SQL/JSON thing is pretty much straight from Oracle and Microsoft (and notably completely different from DB2).

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Peter Geoghegan
On Fri, Jul 1, 2016 at 6:23 AM, Robert Haas wrote: > The proposed patch contains no test case and no description of how to > reproduce the problem. I am not very keen on the idea of trying to > puzzle that out from first principles. I thought that the bug was simple enough that it didn't require

Re: [HACKERS] Broken handling of lwlocknames.h

2016-07-01 Thread Tom Lane
Michael Paquier writes: > Yes that would be indeed cleaner this way. I have poked a bit at that > and finished with the attached that defines some rules to generate all > the files needed. But actually it does not seem to be enough, for > example on OSX this would fail to compile because it cannot

Re: [HACKERS] Bad behavior from plpython 'return []'

2016-07-01 Thread Robert Haas
On Thu, Jun 30, 2016 at 9:25 PM, Jim Nasby wrote: > CREATE FUNCTION pg_temp.bad() RETURNS text[] LANGUAGE plpythonu AS $$return > []$$; > SELECT pg_temp.bad(); > bad > - > {} > (1 row) > > SELECT pg_temp.bad() = '{}'::text[]; > ?column? > -- > f > (1 row) > > Erm?? Turns out this i

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-07-01 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Jul 1, 2016 at 8:50 AM, Michael Paquier > wrote: > >> Okay, that argument I buy. > >> > >> I suppose this function/view should report no row at all if there is no > >> wal receiver connected, rather than a view with nulls. > > > > The function returns PG_RETURN_NU

Re: [HACKERS] Forthcoming SQL standards about JSON and Multi-Dimensional Arrays (FYI)

2016-07-01 Thread Andres Freund
On 2016-07-01 19:06:06 +0800, Craig Ringer wrote: > On 1 July 2016 at 17:52, Oleg Bartunov wrote: > > > On Wed, Jun 29, 2016 at 2:51 AM, Stefan Keller wrote: > > > Hi, > > > > > > FYI: I'd just like to point you to following two forthcoming standard > > > parts from "ISO/IEC JTS 1/SC 32" comitte

[HACKERS] TLC for EXPLAIN ANALYZE (parallel query and loops)

2016-07-01 Thread David G. Johnston
https://www.postgresql.org/docs/9.6/static/using-explain.html Existing... 14.1.2 Explain Analyze [...] """ In some query plans, it is possible for a subplan node to be executed more than once. For example, the inner index scan will be executed once per outer row in the above nested-loop plan. In

Re: [HACKERS] ToDo: API for SQL statement execution other than SPI

2016-07-01 Thread Vibhor Kumar
> On Jul 1, 2016, at 11:39 AM, Robert Haas wrote: > > On Wed, Jun 29, 2016 at 1:55 AM, Pavel Stehule > wrote: >> I am writing two background workers - autoreindex and scheduler. In Both I >> need to execute queries from top level. I had to wrote redundant code >>

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Alvaro Herrera
Merlin Moncure wrote: > It's pretty easy to craft a query where you're on the winning side, > but what's the worst case of doing two pass...is constant folding a > non trivial fraction of planning time? One thing that has been suggested is to re-examine the plan after planning is done, and if exe

Re: [HACKERS] The link to download PostgreSQL 9.6 Beta 2 for Windows X64 is broken (The link downloads Beta 1)

2016-07-01 Thread Bruce Momjian
On Fri, Jul 1, 2016 at 05:12:40PM +0100, Dave Page wrote: > On Fri, Jul 1, 2016 at 4:07 PM, Jean-Pierre Pelletier > wrote: > > The link to download PostgreSQL 9.6 Beta 2 for Windows X64 > > is not working. > > The link does download something, but it's Beta 1. > > > > http://www.enterprisedb.com/

Re: [HACKERS] The link to download PostgreSQL 9.6 Beta 2 for Windows X64 is broken (The link downloads Beta 1)

2016-07-01 Thread Dave Page
On Fri, Jul 1, 2016 at 4:07 PM, Jean-Pierre Pelletier wrote: > The link to download PostgreSQL 9.6 Beta 2 for Windows X64 > is not working. > The link does download something, but it's Beta 1. > > http://www.enterprisedb.com/products-services-training/pgdownload#windows There is an issue with the

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 12:00 PM, Merlin Moncure wrote: > Sure (I didn't put you on that position, just thinking out loud). The > problem with UNION ALL is that it's only safe to do so when you know > for sure the both sides of the partition are non-overlapping. The > author of the query often kn

[HACKERS] The link to download PostgreSQL 9.6 Beta 2 for Windows X64 is broken (The link downloads Beta 1)

2016-07-01 Thread Jean-Pierre Pelletier
The link to download PostgreSQL 9.6 Beta 2 for Windows X64 is not working. The link does download something, but it's Beta 1. http://www.enterprisedb.com/products-services-training/pgdownload#windows Thanks, Jean-Pierre Pelletier

Re: [HACKERS] Actuall row count of Parallel Seq Scan in EXPLAIN ANALYZE .

2016-07-01 Thread David G. Johnston
On Mon, Jun 20, 2016 at 2:54 AM, Masahiko Sawada wrote: > >> QUERY PLAN > >> > >> > - > >> Final

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Stephen Frost
Tom, all, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > On Tue, Jun 21, 2016 at 4:18 PM, Merlin Moncure wrote: > >> explain analyze select * from foo where false or exists (select 1 from > >> bar where good and foo.id = bar.id); -- A > >> explain analyze select * from foo whe

Re: [HACKERS] Documentation fixes for pg_visibility

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 10:09 AM, Amit Kapila wrote: >>> Ah, I see. So your suggestion is to do this job in lazy_scan_heap() >>> when scanning each block, and then to issue a WARNING and clear the >>> visibility map. Indeed that's better. I guess I need to take a closer >>> look at vacuumlazy.c. Se

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Merlin Moncure
On Fri, Jul 1, 2016 at 10:27 AM, Robert Haas wrote: > On Fri, Jul 1, 2016 at 10:20 AM, Merlin Moncure wrote: >> Yeah. Also, even if you could parse out those cases, it's major >> optimization fence. Consider if you have an ORDER BY clause here: >> >> SELECT FROM foo WHERE a OR b ORDER BY c; >>

Re: [HACKERS] fixing consider_parallel for upper planner rels

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 10:52 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jun 30, 2016 at 5:54 PM, Tom Lane wrote: >>> And the point of that is what, exactly? If the only change is that >>> "some restrictions get enforced", I am not clear on why we need such >>> a test mode in cases whe

Re: [HACKERS] fixing consider_parallel for upper planner rels

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 11:09 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jun 30, 2016 at 5:54 PM, Tom Lane wrote: >>> Don't have time to re-read this right now, but maybe tomorrow or >>> Saturday. > >> OK, thanks. > > There's still the extra-word problem here: > > +* If the inpu

Re: [HACKERS] ToDo: API for SQL statement execution other than SPI

2016-07-01 Thread Robert Haas
On Wed, Jun 29, 2016 at 1:55 AM, Pavel Stehule wrote: > I am writing two background workers - autoreindex and scheduler. In Both I > need to execute queries from top level. I had to wrote redundant code > https://github.com/okbob/autoreindex/blob/master/utils.c > autoreindex_execute_sql_command .S

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-07-01 Thread Robert Haas
On Wed, Jun 29, 2016 at 1:38 AM, Ashutosh Bapat wrote: > On Tue, Jun 28, 2016 at 12:52 PM, Etsuro Fujita > wrote: >> On 2016/06/28 15:23, Ashutosh Bapat wrote: >>> >>> The wording "column "whole-row reference ..." doesn't look good. >>> Whole-row reference is not a column. The error context itsel

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 10:11 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 1, 2016 at 9:52 AM, Tom Lane wrote: >>> Maybe, but neither UNION nor UNION ALL would duplicate the semantics >>> of OR, so there's some handwaving here that I missed. > >> SELECT * FROM foo WHERE a = 5 OR a = 4

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 10:20 AM, Merlin Moncure wrote: > Yeah. Also, even if you could parse out those cases, it's major > optimization fence. Consider if you have an ORDER BY clause here: > > SELECT FROM foo WHERE a OR b ORDER BY c; > > ... by pushing inside a union, you're going to be in troub

Re: [HACKERS] fixing consider_parallel for upper planner rels

2016-07-01 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 30, 2016 at 5:54 PM, Tom Lane wrote: >> Don't have time to re-read this right now, but maybe tomorrow or >> Saturday. > OK, thanks. There's still the extra-word problem here: +* If the input rel is marked consider_parallel and there's nothing +

Re: [HACKERS] fixing consider_parallel for upper planner rels

2016-07-01 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 30, 2016 at 5:54 PM, Tom Lane wrote: >> And the point of that is what, exactly? If the only change is that >> "some restrictions get enforced", I am not clear on why we need such >> a test mode in cases where the planner is afraid to put a top Gather on >> the p

Re: [HACKERS] Reviewing freeze map code

2016-07-01 Thread Masahiko Sawada
On Fri, Jul 1, 2016 at 11:12 AM, Amit Kapila wrote: > On Thu, Jun 30, 2016 at 8:10 PM, Masahiko Sawada > wrote: >> On Thu, Jun 30, 2016 at 3:12 PM, Amit Kapila wrote: >>> On Thu, Jun 30, 2016 at 9:13 AM, Andres Freund wrote: On 2016-06-30 08:59:16 +0530, Amit Kapila wrote: > On Wed, J

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Merlin Moncure
On Fri, Jul 1, 2016 at 9:11 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 1, 2016 at 9:52 AM, Tom Lane wrote: >>> Maybe, but neither UNION nor UNION ALL would duplicate the semantics >>> of OR, so there's some handwaving here that I missed. > >> SELECT * FROM foo WHERE a = 5 OR a = 4

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-07-01 Thread Ashutosh Bapat
On Fri, Jul 1, 2016 at 7:45 PM, Robert Haas wrote: > On Tue, Jun 28, 2016 at 8:20 AM, Ashutosh Bapat > wrote: > >> > postgres_fdw resets the search path to pg_catalog while opening > >> > connection > >> > to the server. The reason behind this is explained in deparse.c > >> > > >> > * We assume

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-07-01 Thread Robert Haas
On Tue, Jun 28, 2016 at 8:20 AM, Ashutosh Bapat wrote: >> > postgres_fdw resets the search path to pg_catalog while opening >> > connection >> > to the server. The reason behind this is explained in deparse.c >> > >> > * We assume that the remote session's search_path is exactly >> > "pg_catalog"

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 1, 2016 at 9:52 AM, Tom Lane wrote: >> Maybe, but neither UNION nor UNION ALL would duplicate the semantics >> of OR, so there's some handwaving here that I missed. > SELECT * FROM foo WHERE a = 5 OR a = 4 > isn't equivalent to > SELECT * FROM foo WHERE a = 5 >

Re: [HACKERS] Documentation fixes for pg_visibility

2016-07-01 Thread Amit Kapila
On Fri, Jul 1, 2016 at 6:58 PM, Robert Haas wrote: > On Wed, Jun 29, 2016 at 1:42 AM, Michael Paquier > wrote: >> On Tue, Jun 28, 2016 at 7:05 AM, Robert Haas wrote: >>> On Mon, Jun 27, 2016 at 5:56 PM, Michael Paquier >>> wrote: > Under what circumstances would you wish to check only one p

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 9:52 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jun 21, 2016 at 4:18 PM, Merlin Moncure wrote: >>> explain analyze select * from foo where false or exists (select 1 from >>> bar where good and foo.id = bar.id); -- A >>> explain analyze select * from foo where ex

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-01 Thread Kevin Grittner
On Fri, Jul 1, 2016 at 7:17 AM, Robert Haas wrote: > On Fri, Jul 1, 2016 at 2:48 AM, Andres Freund wrote: >>> This PostgreSQL 9.6 open item is past due for your status update. Kindly >>> send >>> a status update within 24 hours, and include a date for your subsequent >>> status >>> update. Re

Re: [HACKERS] how is the WAL receiver process stopped and restarted when the network connection is broken and then restored?

2016-07-01 Thread Robert Haas
On Thu, Jun 23, 2016 at 10:56 AM, Rui Hai Jiang wrote: > Please let me know if my understanding is incorrect. I think you've got it about right. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] Extract Jsonb key and values

2016-07-01 Thread Robert Haas
On Thu, Jun 23, 2016 at 8:52 AM, hari.prasath wrote: > Hi all, > I am having jsonb as C character string received by WAL decoding and > want to extract all its key and value pairs. > > >Which is the best approach for extracting keys and its values? > > i) Converting the C string to a

Re: [HACKERS] Comment and function argument names are mismatched in bugmgr.c.

2016-07-01 Thread Robert Haas
On Thu, Jun 23, 2016 at 3:24 AM, Masahiko Sawada wrote: > By commit 428b1d6b, function WritebackContextInit is added but the > comment of this function seems to be incorrect. > *max_coalesce variable doesn't exist at anywhere. > Also, I think it should be fixed that the argument name of this > fun

Re: [HACKERS] Actuall row count of Parallel Seq Scan in EXPLAIN ANALYZE .

2016-07-01 Thread Robert Haas
On Mon, Jun 20, 2016 at 3:06 AM, Satoshi Nagayasu wrote: > IMHO, "actual rows" of "Parallel Seq Scan" should not be divided by the loops, > because the total rows processed here is 1000, not 333 * 3. > I think the actual row value shown here "333 " is a bit confusing and > tricky > fo

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Tom Lane
Robert Haas writes: > On Tue, Jun 21, 2016 at 4:18 PM, Merlin Moncure wrote: >> explain analyze select * from foo where false or exists (select 1 from >> bar where good and foo.id = bar.id); -- A >> explain analyze select * from foo where exists (select 1 from bar >> where good and foo.id = bar.

Re: [HACKERS] Truncating/vacuuming relations on full tablespaces

2016-07-01 Thread Robert Haas
On Mon, Jun 20, 2016 at 7:28 AM, Asif Naeem wrote: > Thank you for useful suggestions. PFA patch, I have tried to cover all the > points mentioned. Thanks for the new patch. I think that you have failed to address this point from my previous review: # I see why you changed the calling conventio

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Robert Haas
On Tue, Jun 21, 2016 at 4:18 PM, Merlin Moncure wrote: > Observe the following test case (apologies if this is a well > understood problem): > > create temp table foo as select generate_series(1,100) id; > create index on foo(id); > > create temp table bar as select id, id % 10 = 0 as good

Re: [HACKERS] Broken handling of lwlocknames.h

2016-07-01 Thread Michael Paquier
On Fri, Jul 1, 2016 at 10:29 PM, Tom Lane wrote: > Robert Haas writes: >> Since Tom proposed the approach which Michael's patch takes, I'm >> hoping he will review and commit this. If it is left to me to fix it, >> I may just adopt a minimal fix. > > I'll take a look at it. Note: the patch is n

Re: [HACKERS] Broken handling of lwlocknames.h

2016-07-01 Thread Tom Lane
Robert Haas writes: > Since Tom proposed the approach which Michael's patch takes, I'm > hoping he will review and commit this. If it is left to me to fix it, > I may just adopt a minimal fix. I'll take a look at it. regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] Documentation fixes for pg_visibility

2016-07-01 Thread Robert Haas
On Wed, Jun 29, 2016 at 1:42 AM, Michael Paquier wrote: > On Tue, Jun 28, 2016 at 7:05 AM, Robert Haas wrote: >> On Mon, Jun 27, 2016 at 5:56 PM, Michael Paquier >> wrote: Under what circumstances would you wish to check only one page of a relation? >>> >>> What I'd like to be able to

Re: [HACKERS] Bug in batch tuplesort memory CLUSTER case (9.6 only)

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 12:06 AM, Noah Misch wrote: > On Sun, Jun 26, 2016 at 09:14:05PM -0700, Peter Geoghegan wrote: >> In general, moving tuplesort.c batch memory caller tuples around >> happens when batch memory needs to be recycled, or freed outright with >> pfree(). >> >> I failed to take int

Re: [HACKERS] Broken handling of lwlocknames.h

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 12:09 AM, Noah Misch wrote: > [Action required within 72 hours. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 9.6 open item. Robert, > since you committed the patch believed to have created it, you own this open > item. If some o

Re: [HACKERS] [sqlsmith] crashes in RestoreSnapshot on hot standby

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 1:41 AM, Andreas Seltenreich wrote: > Amit Kapila writes: >> On Fri, Jul 1, 2016 at 9:38 AM, Thomas Munro >> wrote: >>> Or maybe just like this? >>> >>> - snapshot->subxip = snapshot->xip + >>> serialized_snapshot->xcnt; >>> + snapshot->subxip

[HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-01 Thread Robert Haas
On Fri, Jul 1, 2016 at 2:48 AM, Andres Freund wrote: >> This PostgreSQL 9.6 open item is past due for your status update. Kindly >> send >> a status update within 24 hours, and include a date for your subsequent >> status >> update. Refer to the policy on open item ownership: >> http://www.pos

[HACKERS] Re: [ADMIN] problems using pg_start_backup/pg_stop_backup and pg_basebackup at same time

2016-07-01 Thread Magnus Hagander
On Mon, Jun 27, 2016 at 8:27 PM, Alex Malek wrote: > Should I file a bug? > Oh, sorry. I did not spot that this was posted to pgsql-admin, which has a lot fewer readers. I've moved it over to -hackers where more people will see it. (And also, please don't top-post on these lists, as it makes th

Re: [HACKERS] Forthcoming SQL standards about JSON and Multi-Dimensional Arrays (FYI)

2016-07-01 Thread Craig Ringer
On 1 July 2016 at 17:52, Oleg Bartunov wrote: > On Wed, Jun 29, 2016 at 2:51 AM, Stefan Keller wrote: > > Hi, > > > > FYI: I'd just like to point you to following two forthcoming standard > > parts from "ISO/IEC JTS 1/SC 32" comittee: one on JSON, and one on > > "Multi-Dimensional Arrays" (SQL/M

Re: [HACKERS] Forthcoming SQL standards about JSON and Multi-Dimensional Arrays (FYI)

2016-07-01 Thread Oleg Bartunov
On Wed, Jun 29, 2016 at 2:51 AM, Stefan Keller wrote: > Hi, > > FYI: I'd just like to point you to following two forthcoming standard > parts from "ISO/IEC JTS 1/SC 32" comittee: one on JSON, and one on > "Multi-Dimensional Arrays" (SQL/MDA). > > They define there some things different as already

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Christoph Berg
Re: Andreas Karlsson 2016-07-01 <688a438c-ccc2-0431-7100-26e418fc3...@proxel.se> > Hi, > > Here is an initial set of patches related to OpenSSL 1.1. Everything should > still build fine on older OpenSSL versions (and did when I tested with > 1.0.2h). Hi Andreas, thanks for the patches. I applied

Re: [HACKERS] WIP: About CMake v2

2016-07-01 Thread Dmitry Maslyuk
Hi! I think, we need simple configure script generator for backward compatibility and easy using this build system. Try playing with cmake build system under Win2008+MinGW. I plan to write perl script for automatic build this with depends. On 29.06.2016 19:23, Yury Zhuravlev wrote: Hello Hac

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Magnus Hagander
On Fri, Jul 1, 2016 at 10:10 AM, Michael Paquier wrote: > On Fri, Jul 1, 2016 at 5:02 PM, Magnus Hagander > wrote: > > Debian testing is still on 1.0.2h. > > Debian experimental is on 1.1.0pre5. > > > > Not sure here beta2 enters the discussion, it's not mentioned anywhere on > > their site? > >

Re: [HACKERS] WIP: About CMake v2

2016-07-01 Thread Yury Zhuravlev
Michael Paquier wrote: Personally I am allergic to any kind of UIs for development, and I am sure not to be the only one on this list. Andrew Dunstan: We need this to be scriptable, not using a GUI. GUI is strong optional feature. Helpful for some tasks. -- Yury Zhuravlev Postgres Profess

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Michael Paquier
On Fri, Jul 1, 2016 at 5:02 PM, Magnus Hagander wrote: > Debian testing is still on 1.0.2h. > Debian experimental is on 1.1.0pre5. > > Not sure here beta2 enters the discussion, it's not mentioned anywhere on > their site? Thanks. From the main page of openssl.org, pre5 is beta2. -- Michael --

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Magnus Hagander
On Fri, Jul 1, 2016 at 4:08 AM, Michael Paquier wrote: > On Fri, Jul 1, 2016 at 9:27 AM, Andreas Karlsson > wrote: > > Hi, > > > > Here is an initial set of patches related to OpenSSL 1.1. Everything > should > > still build fine on older OpenSSL versions (and did when I tested with > > 1.0.2h).

Re: [HACKERS] Is a UDF binary portable across different minor releases and PostgreSQL distributions?

2016-07-01 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Craig Ringer There's no formal extension API. So there's no boundary between "internal stuff we might have to change to fix a problem" and "things extensions can rely on not changing under them". In

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-07-01 Thread Etsuro Fujita
On 2016/04/14 4:57, Robert Haas wrote: 1. For a regular FDW scan, zero the xmin, xmax, and cid of the tuple before returning it from postgres_fdw, so that we don't expose the datum-tuple fields. I can't see any reason this isn't safe, but I might be missing something. I noticed odd behavior o