Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-03-09 Thread Amit Kapila
On Sat, Mar 10, 2018 at 1:10 PM, Amit Kapila wrote: > On Tue, Mar 6, 2018 at 11:26 AM, Amit Kapila wrote: >> On Mon, Mar 5, 2018 at 8:58 AM, Thomas Munro >> wrote: >>> On Sun, Mar 4, 2018 at 12:53 AM, Amit Kapila >>> wrote: Yes, but I think it would be better if we call this once we are s

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-03-09 Thread Amit Kapila
On Tue, Mar 6, 2018 at 11:26 AM, Amit Kapila wrote: > On Mon, Mar 5, 2018 at 8:58 AM, Thomas Munro > wrote: >> On Sun, Mar 4, 2018 at 12:53 AM, Amit Kapila wrote: >>> Yes, but I think it would be better if we call this once we are sure >>> that at least one tuple from the old bucket has been tra

Re: csv format for psql

2018-03-09 Thread Fabien COELHO
recordsep in the unaligned mode doesn't play the role of a line ending because the last line is not finished by recordsep. According to the source code, this is intended, see print_unaligned_text() in print.c: Something else comes to mind: CSV allows linefeeds inside fields, and we don't want

Re: csv format for psql

2018-03-09 Thread Fabien COELHO
Hello Daniel, About "-C", I'm fine if it is used and if it is not used, really. psql begins to be quite full of one letter options, currently 34 of them, with upper & lower cases and numbers included. The solution to set fieldsep automatically to ',' with \pset format csv is problematic.

Re: constraint exclusion and nulls in IN (..) clause

2018-03-09 Thread Tom Lane
I wrote: > I think it'd make more sense to see about incorporating that idea in > predicate_implied_by_simple_clause/predicate_refuted_by_simple_clause. After further thought, it seems like the place to deal with this is really operator_predicate_proof(), as in the attached draft patch against HEA

Re: csv format for psql

2018-03-09 Thread David G. Johnston
On Fri, Mar 9, 2018 at 3:18 PM, Daniel Verite wrote: > I think that the point of recordsep in unaligned mode is you can set it > to something that never appears in the data, especially when embedded > newlines might be in the data. In CSV this is solved differently so > we don't need it. ​I'd r

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2018-03-09 Thread Yura Sokolov
08.03.2018 03:42, Tomas Vondra пишет: > On 03/06/2018 06:23 AM, Yura Sokolov wrote: >> 05.03.2018 18:00, Tom Lane пишет: >>> Tomas Vondra writes: Snapshots are static (we don't really add new XIDs into existing ones, right?), so why don't we simply sort the XIDs and then use bsearch to >

Bogus use of canonicalize_qual

2018-03-09 Thread Tom Lane
Whilst fooling about with predtest.c, I noticed a rather embarrassing error. Consider the following, rather silly, CHECK constraint: regression=# create table pp (f1 int); CREATE TABLE regression=# create table cc (check (f1 = 1 or f1 = null)) inherits(pp); CREATE TABLE Because "f1 = null" reduc

Re: PATCH: Unlogged tables re-initialization tests

2018-03-09 Thread Peter Eisentraut
This seems like a useful test. On 3/5/18 12:35, David Steele wrote: > +mkdir($tablespaceDir) > + or die "unable to mkdir \"$tablespaceDir\""; Use BAIL_OUT instead of die in tests. > + $ts1UnloggedPath = $node->safe_psql('postgres', > + q{select pg_relation_filepath('ts1_unlogged')});

Re: csv format for psql

2018-03-09 Thread Daniel Verite
David G. Johnston wrote: > I'm not following - if recordsep is not something that would > interpreted as a newline then the file we output would have not > structural newlines. > It might have data newlines but those would be quoted. They would be, but I don't quite understand the point i

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-09 Thread Peter Eisentraut
On 1/13/18 22:19, Haribabu Kommi wrote: > While working on [1], we find out the inconsistency in PQHost() behavior > if the connecting string that is passed to connect to the server contains > multiple hosts with both host and hostaddr types. For example, > > host=host1,host2 hostaddr=127.0.0.1,12

Re: [PATCH] Verify Checksums during Basebackups

2018-03-09 Thread Michael Banck
Hi, Am Mittwoch, den 28.02.2018, 19:08 +0100 schrieb Michael Banck: > some installations have data which is only rarerly read, and if they are > so large that dumps are not routinely taken, data corruption would only > be detected with some large delay even with checksums enabled. > > The attache

Re: JIT compiling with LLVM v11

2018-03-09 Thread Andres Freund
On 2018-03-09 15:42:24 -0500, Peter Eisentraut wrote: > For jit_optimize_above_cost, in my testing, any query where JIT payed > off was even faster with optimizing. So right now I don't see a need to > make this a separate setting. Maybe just make it an on/off setting for > experimenting. I'd pr

Re: FOR EACH ROW triggers on partitioned tables

2018-03-09 Thread Alvaro Herrera
... and this little addendum makes pg_dump work correctly. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 867bbe8f1e..ca0a66753e 100644

Re: using worker_spi as pattern

2018-03-09 Thread Jeremy Finzel
On Fri, Mar 9, 2018 at 12:34 AM, Michael Paquier wrote: > On Thu, Mar 08, 2018 at 11:04:20PM -0600, Jeremy Finzel wrote: > > Since you mention, can anyone elaborate further on the memory leak danger > > here? > > > > Line 193 in src/test/modules/worker_spi/worker_spi.c read: > > # Note some memor

Re: JIT compiling with LLVM v11

2018-03-09 Thread Andres Freund
On 2018-03-09 15:28:19 -0500, Peter Eisentraut wrote: > On 3/6/18 15:16, Andres Freund wrote: > > 2) Don't load the JIT provider until fully needed. Right now > >jit_compile_expr() will load the jit provider even if not really > >needed. We should probably move the first two return blocks i

Re: JIT compiling with LLVM v11

2018-03-09 Thread Peter Eisentraut
On 3/6/18 10:29, Peter Eisentraut wrote: > I think taking the total cost as the triggering threshold is probably > good enough for a start. The cost modeling can be refined over time. I looked into this a bit more. The default of jit_above_cost = 50 seems pretty good. I constructed a query

Re: FOR EACH ROW triggers on partitioned tables

2018-03-09 Thread Alvaro Herrera
Alvaro Herrera wrote: > One thing I'd like to add before claiming this committable (backend- > side) is enabling constraint triggers. AFAICT that requires a bit of > additional logic, but it shouldn't be too terrible. This would allow > for deferrable unique constraints, for example. v7 support

Re: JIT compiling with LLVM v11

2018-03-09 Thread Peter Eisentraut
On 3/6/18 15:16, Andres Freund wrote: > 2) Don't load the JIT provider until fully needed. Right now >jit_compile_expr() will load the jit provider even if not really >needed. We should probably move the first two return blocks in >llvm_compile_expr() into jit_compile_expr(), to avoid t

Re: Implementing SQL ASSERTION

2018-03-09 Thread Thomas Munro
On Sat, Mar 10, 2018 at 6:37 AM, Robert Haas wrote: > On Mon, Jan 15, 2018 at 11:35 AM, David Fetter wrote: >> - We follow the SQL standard and make SERIALIZABLE the default >> transaction isolation level, and > > The consequences of such a decision would include: > > - pgbench -S would run up

Re: FOR EACH ROW triggers on partitioned tables

2018-03-09 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 3/7/18 20:57, Alvaro Herrera wrote: > > So, unless someone has a brilliant idea on how to construct a column > > mapping from partitioned table to partition, I'm going back to the > > design I was proposing earlier, ie., creating individual pg_trigger rows > > for each

Re: csv format for psql

2018-03-09 Thread David G. Johnston
On Fri, Mar 9, 2018 at 12:42 PM, Daniel Verite wrote: > I wrote: > > > recordsep in the unaligned mode doesn't play the role of a line ending > > because the last line is not finished by recordsep. According to the > source > >code, this is intended, see print_unaligned_text() in print.c:

Re: csv format for psql

2018-03-09 Thread Daniel Verite
I wrote: > recordsep in the unaligned mode doesn't play the role of a line ending > because the last line is not finished by recordsep. According to the source >code, this is intended, see print_unaligned_text() in print.c: Something else comes to mind: CSV allows linefeeds inside fields,

Re: csv format for psql

2018-03-09 Thread Daniel Verite
Fabien COELHO wrote: > I also think that a short option brings little value, and "--csv" is good > enough for the purpose, so I would agree to remove the "-C" binding. It's not that accepting -C brings much value by itself, it's that loosing the consistency across all options comes with

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-09 Thread Arthur Zakirov
Greetings, On Thu, Mar 08, 2018 at 02:58:34PM +0100, Julian Markwort wrote: > > I'd love to hear more feedback, here are two ideas to polish this > > patch: > > a) Right now, good_plan and bad_plan collection can be activated and > > deactivated with separate GUCs. I think it would be sensible to

Re: PATCH: Configurable file mode mask

2018-03-09 Thread David Steele
Hi Michael, On 3/7/18 8:51 PM, Michael Paquier wrote: > On Wed, Mar 07, 2018 at 10:56:32AM -0500, David Steele wrote: >> On 3/6/18 10:04 PM, Michael Paquier wrote: >>> Seems like you forgot to re-add the chmod calls in initdb.c. >> >> Hmmm, I thought we were talking about moving the position of um

Re: [HACKERS] Runtime Partition Pruning

2018-03-09 Thread Jesper Pedersen
Hi David, On 03/01/2018 08:29 PM, David Rowley wrote: 0004 fails "make check-world" due to pg_restore: [archiver (db)] Error while PROCESSING TOC: pg_restore: [archiver (db)] Error from TOC entry 670; 1259 49954 TABLE boolp_f jpedersen pg_restore: [archiver (db)] could not execute query: ERROR:

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-03-09 Thread Alexander Korotkov
On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada wrote: > On Fri, Mar 9, 2018 at 8:43 AM, Alexander Korotkov > wrote: > > 2) These parameters are reset during btbulkdelete() and set during > > btvacuumcleanup(). > > Can't we set these parameters even during btbulkdelete()? By keeping > them up to

Re: Faster inserts with mostly-monotonically increasing values

2018-03-09 Thread Claudio Freire
On Fri, Mar 9, 2018 at 2:54 PM, Pavan Deolasee wrote: > > > On Tue, Mar 6, 2018 at 10:10 AM, Pavan Deolasee > wrote: >> >> >> >> On Tue, Mar 6, 2018 at 7:29 AM, Peter Geoghegan wrote: >>> >>> On Mon, Mar 5, 2018 at 5:48 PM, Claudio Freire >>> wrote: >>> >>> > I believe PKs are a prime candidate

Re: Failed to request an autovacuum work-item in silence

2018-03-09 Thread Masahiko Sawada
Thank you for reviewing! On Thu, Mar 8, 2018 at 6:07 PM, Ildar Musin wrote: > Just couple remarks. I would rename 'requested' variable in > AutoVacuumRequestWork() func to something like 'success' or 'result'. > Because request is something caller does. And I would also rephrase log > message as

Re: Faster inserts with mostly-monotonically increasing values

2018-03-09 Thread Pavan Deolasee
On Tue, Mar 6, 2018 at 10:10 AM, Pavan Deolasee wrote: > > > On Tue, Mar 6, 2018 at 7:29 AM, Peter Geoghegan wrote: > >> On Mon, Mar 5, 2018 at 5:48 PM, Claudio Freire >> wrote: >> >> > I believe PKs are a prime candidate for this optimization, and >> > expecting it to apply only when no concur

Re: Implementing SQL ASSERTION

2018-03-09 Thread Robert Haas
On Mon, Jan 15, 2018 at 11:35 AM, David Fetter wrote: > - We follow the SQL standard and make SERIALIZABLE the default > transaction isolation level, and The consequences of such a decision would include: - pgbench -S would run up to 10x slower, at least if these old benchmark results are stil

Re: Precision loss casting float to numeric

2018-03-09 Thread Emre Hasegeli
>> I wonder if an alternative to making a cast that can't be immutable, >> because it looks at a GUC, could be to offer a choice of cast >> functions: if you need the other behavior, create a schema, do a >> CREATE CAST in it, and put it on your search path ahead of pg_catalog. > > Nope, because ca

Re: csv format for psql

2018-03-09 Thread Pavel Stehule
2018-03-09 8:40 GMT+01:00 Fabien COELHO : > > About "fieldsep_csv", I do not like much the principle of having different output variables to represent the same concept depending on the format. I would rather have reused fieldsep as in your previous submission and set it to "," when

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-09 Thread Peter Eisentraut
I think this patch is not going to be ready for PG11. - It depends on some work in the thread "logical decoding of two-phase transactions", which is still in progress. - Various details in the logical_work_mem patch (0001) are unresolved. - This being partially a performance feature, we haven't

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-09 Thread Peter Eisentraut
On 2/1/18 19:21, Simon Riggs wrote: > If we really can't persuade you of that, it doesn't sink the patch. We > can have the WAL pointer itself - it wouldn't save space but it would > at least alleviate the spinlock. Do you want to send in an alternative patch that preserves the WAL pointer and onl

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-09 Thread Alvaro Herrera
Please don't forget to remove the xlog.c and miscadmin.h hunks from the patch. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: using index or check in ALTER TABLE SET NOT NULL

2018-03-09 Thread Sergei Kornilov
Hello all! Summarizing, how i must update tests? We want test real skip of verify phase or only result correctness? I can verify what i do not break alter table by tests in v4 of my patch. But here is no check code path. So my feature may be broken in future without test fail. alter table will b

Re: Testbed for predtest.c ... and some arguable bugs therein

2018-03-09 Thread Tom Lane
Robert Haas writes: > The use of the terms "refutes", "non-truth", and "non-falsity" drive > me a little batty; they've all got an internal negation built into > them, and that's confusing. It seems very tempting to propose getting > rid of predicate_refuted_by altogether and to simply have one >

Re: Fixes for missing schema qualifications

2018-03-09 Thread David Steele
On 3/9/18 2:55 AM, Michael Paquier wrote: > > In light of CVE-2018-1058, user's applications need to be careful about > the use of schema-unqualified queries. A lookup at the upstream code is > showing four areas which are missing such handling: > - psql has one problem in get_create_object_cmd w

Re: disable SSL compression?

2018-03-09 Thread Magnus Hagander
On Fri, Mar 9, 2018 at 3:06 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/8/18 14:23, Claudio Freire wrote: > > On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut > > wrote: > >> It appears that SSL compression is nowadays deprecated as insecure. > >> Yet, it is still enabl

Re: FOR EACH ROW triggers on partitioned tables

2018-03-09 Thread Peter Eisentraut
On 3/7/18 20:57, Alvaro Herrera wrote: > So, unless someone has a brilliant idea on how to construct a column > mapping from partitioned table to partition, I'm going back to the > design I was proposing earlier, ie., creating individual pg_trigger rows > for each partition that are essentially adj

Re: [PATCH] pg_hba.conf : new auth option : clientcert=verify-full

2018-03-09 Thread Julian Markwort
Hello Magnus, > I think this makes a lot of sense, and can definitely be a useful > option. I was hesistant to write a long and elaborate patch as I wasn't certain if there was any interest for such an addition, but I'm thankful for your input. > However, the patch is completely lacking document

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-03-09 Thread Masahiko Sawada
On Fri, Mar 9, 2018 at 8:43 AM, Alexander Korotkov wrote: > Hi! > Sorry for my late reply. > I'd like to propose a revised patch based on various ideas upthread. Thank you for proposing the patch! > > This patch works as following. > > 1) B-tree meta page is extended with 2 additional paramete

Re: Testbed for predtest.c ... and some arguable bugs therein

2018-03-09 Thread Robert Haas
On Thu, Mar 8, 2018 at 6:24 PM, Tom Lane wrote: > * R1: A refutes B if truth of A implies falsity of B. > * R2: A refutes B if truth of A implies non-truth of B. > * R3: A refutes B if non-falsity of A implies non-truth of B. > * R4: A refutes B if non-falsity of A implies falsity of B. The use o

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-09 Thread Etsuro Fujita
Hi Ashutosh, (2018/03/08 14:24), Ashutosh Bapat wrote: Etsuro said [2] that WCO constraints can not be implemented on foreign server and normal check constraints can be, and for that he provides an example in [3]. But I think that example is going the wrong direction. More precisely, what I'm

Re: inserts into partitioned table may cause crash

2018-03-09 Thread Etsuro Fujita
(2018/03/06 21:26), Etsuro Fujita wrote: One thing I notice while working on this is this in ExecInsert/CopyFrom, which I moved to ExecPrepareTupleRouting as-is for the former: /* * If we're capturing transition tuples, we might need to convert from the * partition rowtype to parent rowtype. */

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-09 Thread Ashutosh Bapat
On Thu, Mar 8, 2018 at 7:31 PM, Robert Haas wrote: > > This kind of goes along with the suggestion I made yesterday to > introduce a new function, which at the time I proposed calling > initialize_grouping_rel(), to set up new grouped or partially grouped > relations. By doing that it would be ea

Re: [PATCH] Improve geometric types

2018-03-09 Thread Emre Hasegeli
> Thank you for the revised version. This seems fine for me so > marked this as "Ready for Committer". Thank you for bearing with me for longer than year. > By the way I think that line_perp is back patched, could you > propose it for the older versions? (I already marked my entry as > Rejected)

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-09 Thread amul sul
On Thu, Mar 8, 2018 at 12:31 PM, Amit Kapila wrote: > On Thu, Mar 8, 2018 at 11:04 AM, Pavan Deolasee > wrote: >> >> On Tue, Feb 13, 2018 at 12:41 PM, amul sul wrote: >>> >>> Thanks for the confirmation, updated patch attached. >>> >> >> I am actually very surprised that 0001-Invalidate-ip_blkid

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-09 Thread amul sul
Hi Andres, Thanks for your time and the review comments/suggestions. On Tue, Mar 6, 2018 at 4:53 AM, Andres Freund wrote: > Hi, > > On 2018-02-13 12:41:26 +0530, amul sul wrote: >> From 08c8c7ece7d9411e70a780dbeed89d81419db6b6 Mon Sep 17 00:00:00 2001 >> From: Amul Sul >> Date: Tue, 13 Feb 2018

Re: Protect syscache from bloating with negative cache entries

2018-03-09 Thread Kyotaro HORIGUCHI
At Wed, 07 Mar 2018 23:12:29 -0500, Tom Lane wrote in <352.1520482...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > At Thu, 8 Mar 2018 00:28:04 +, "Tsunakawa, Takayuki" > > wrote in > > <0A3221C70F24FB45833433255569204D1F8FF0D9@G01JPEXMBYT05> > >> Yes. We are now facing the problem of t

RE: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-09 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:mich...@paquier.xyz] > I see. So the only reason why this flag exists is that if a file is large > enough so as it is split into multiple chunks, then the first unlink will > work but not the successive ones. One chunk can be at most > 1 million bytes, which is why i

Problem while setting the fpw with SIGHUP

2018-03-09 Thread Dilip Kumar
While setting the full_page_write with SIGHUP I hit an assert in checkpoint process. And, that is because inside a CRITICAL section we are calling RecoveryInProgress which intern allocates memory. So I have moved RecoveryInProgress call out of the CRITICAL section and the problem got solved. comm

Re: [PATCH] Improve geometric types

2018-03-09 Thread Kyotaro HORIGUCHI
Hello, sorry to be late. At Fri, 2 Mar 2018 15:26:25 +0100, Emre Hasegeli wrote in > > Unfortunately according to http://commitfest.cputube.org/ this patch > > doesn't apply anymore. > New versions are attached. Thank you for the revised version. This seems fine for me so marked this as "Read