Fixes for missing schema qualifications

2018-03-08 Thread Michael Paquier
Hi all, In light of CVE-2018-1058, user's applications need to be careful about the use of schema-unqualified queries. A lookup at the upstream code is showing four areas which are missing such handling: - psql has one problem in get_create_object_cmd which misses twice to qualify array_remove().

Re: csv format for psql

2018-03-08 Thread Fabien COELHO
About "fieldsep_csv", I do not like much the principle of having different output variables to represent the same concept depending on the format. I would rather have reused fieldsep as in your previous submission and set it to "," when under --csv. yes how will be possible to set differe

Re: public schema default ACL

2018-03-08 Thread Noah Misch
On Thu, Mar 08, 2018 at 02:00:23PM -0500, Robert Haas wrote: > I also wonder why we're all convinced that this urgently needs to be > changed. I agree that the default configuration we ship is not the > most secure configuration that we could ship. However, I think it's a > big step from saying t

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-08 Thread Michael Paquier
On Thu, Mar 01, 2018 at 01:26:32AM +, Tsunakawa, Takayuki wrote: > Our customer hit another bug of pg_rewind with PG 9.5. The attached > patch fixes this. Sorry for my late reply. It took me unfortunately some time before coming back to this patch. > PROBLEM > ==

Re: Let's remove DSM_INPL_NONE.

2018-03-08 Thread Kyotaro HORIGUCHI
At Wed, 28 Feb 2018 10:08:59 +0900, Michael Paquier wrote in <20180228010859.gb1...@paquier.xyz> > On Tue, Feb 27, 2018 at 02:53:55PM -0500, Tom Lane wrote: > > Andres Freund writes: > >> On 2018-02-27 14:41:47 -0500, Tom Lane wrote: > >>> What I didn't understand about it was what kind of testi

Re: public schema default ACL

2018-03-08 Thread Noah Misch
On Wed, Mar 07, 2018 at 07:14:43AM -0500, Stephen Frost wrote: > * Noah Misch (n...@leadboat.com) wrote: > > I like the idea of getting more SQL-compatible, if this presents a distinct > > opportunity to do so. I do think it would be too weird to create the schema > > in one database only. Creati

Re: public schema default ACL

2018-03-08 Thread Noah Misch
On Wed, Mar 07, 2018 at 09:22:16AM -0500, Peter Eisentraut wrote: > On 3/6/18 15:20, Robert Haas wrote: > > On Sat, Mar 3, 2018 at 4:56 AM, Noah Misch wrote: > >> I propose, for v11, switching to "GRANT USAGE ON SCHEMA > >> public TO PUBLIC" (omit CREATE). Concerns? An alternative is to change

Re: using worker_spi as pattern

2018-03-08 Thread Michael Paquier
On Thu, Mar 08, 2018 at 11:04:20PM -0600, Jeremy Finzel wrote: > Since you mention, can anyone elaborate further on the memory leak danger > here? > > Line 193 in src/test/modules/worker_spi/worker_spi.c read: > # Note some memory might be leaked here. > > Is this any reason *not *to use this pat

Re: disable SSL compression?

2018-03-08 Thread Gasper Zejn
On 09. 03. 2018 06:24, Craig Ringer wrote: > I'm totally unconvinced by the threat posed by exploiting a client by > tricking it into requesting protocol compression - or any other > protocol change the client lib doesn't understand - with a connection > option in PGOPTIONS or the "options" connstr

Re: disable SSL compression?

2018-03-08 Thread Claudio Freire
On Thu, Mar 8, 2018 at 11:06 PM, Peter Eisentraut wrote: > On 3/8/18 14:23, Claudio Freire wrote: >> On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut >> wrote: >>> It appears that SSL compression is nowadays deprecated as insecure. >>> Yet, it is still enabled by libpq by default, and there is no

Re: disable SSL compression?

2018-03-08 Thread Craig Ringer
On 9 March 2018 at 03:23, Claudio Freire wrote: > On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut > wrote: > > It appears that SSL compression is nowadays deprecated as insecure. > > Yet, it is still enabled by libpq by default, and there is no way to > > disable it in the server. Should we ma

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-08 Thread Ashutosh Bapat
On Thu, Mar 8, 2018 at 8:02 PM, Jeevan Chalke wrote: > > > On Thu, Mar 8, 2018 at 7:49 PM, Robert Haas wrote: >> >> On Thu, Mar 8, 2018 at 9:15 AM, Jeevan Chalke >> wrote: >> > I am not sure why we don't set reltarget into the grouped_rel too. >> > >> > But if we do so like we did in partially_g

Re: using worker_spi as pattern

2018-03-08 Thread Jeremy Finzel
> > If you look at the code of worker_spi.c closely the answer shows up by > itself: > > appendStringInfo(&buf, > "CREATE SCHEMA \"%s\" " > "CREATE TABLE \"%s\" (" > " type text CHECK (type IN ('total', > 'delta')), " >

Re: csv format for psql

2018-03-08 Thread Pavel Stehule
2018-03-09 3:13 GMT+01:00 Peter Eisentraut : > On 3/8/18 05:05, Fabien COELHO wrote: > > I'm in favor of having a simple psql way to generate a convenient and > > compliant CSV output for export/import. > > yes > > > I also think that a short option brings little value, and "--csv" is good > > eno

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-08 Thread Peter Geoghegan
On Thu, Mar 8, 2018 at 4:54 AM, Pavan Deolasee wrote: > Thanks for the feedback. I've greatly refactored the code based on your > comments and I too like the resultant code. What we have now have > essentially is: two functions: I really like how ExecMerge() now mostly just consists of this code

Re: csv format for psql

2018-03-08 Thread Peter Eisentraut
On 3/8/18 05:05, Fabien COELHO wrote: > I'm in favor of having a simple psql way to generate a convenient and > compliant CSV output for export/import. yes > I also think that a short option brings little value, and "--csv" is good > enough for the purpose, so I would agree to remove the "-C" b

Re: disable SSL compression?

2018-03-08 Thread Peter Eisentraut
On 3/8/18 14:23, Claudio Freire wrote: > On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut > wrote: >> It appears that SSL compression is nowadays deprecated as insecure. >> Yet, it is still enabled by libpq by default, and there is no way to >> disable it in the server. Should we make some change

RE: Speed up the removal of WAL files

2018-03-08 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:mich...@paquier.xyz] > Hm. durable_xx should remain a sane operation as an isolated call as you > still get the same problem if a crash happens before flushing the parent... > Fujii-san idea also has value to speed up the end of recovery but this costs > as well in ex

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-08 Thread Peter Geoghegan
On Thu, Mar 8, 2018 at 3:29 PM, Tomas Vondra wrote: > The reason why the patch tried to prevent that is because the SQL > standard says this (p. 1176 of SQL 2016): > > 15) The immediately contained in a , > the immediately contained in a clause>, and the immediately contained in a when not ma

Re: Speed up the removal of WAL files

2018-03-08 Thread Michael Paquier
On Wed, Mar 07, 2018 at 06:15:24AM +, Tsunakawa, Takayuki wrote: > Good point. I understood you referred to PreallocXlogFiles(), which > may create one new WAL file if RemoveNonParentXlogFiles() is not > called or does not recycle WAL files in the old timeline. > > The best hack (or a compro

Re: using worker_spi as pattern

2018-03-08 Thread Michael Paquier
On Thu, Mar 08, 2018 at 03:29:52PM -0600, Jeremy Finzel wrote: > However, this raises many questions for me: > >- Insert a value into what table? I see the process referring to an >object that doesn't exist in my database - schema1.counted >- What is "total" type? I don't see any typ

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-03-08 Thread Alexander Korotkov
Hi! I'd like to propose a revised patch based on various ideas upthread. This patch works as following. 1) B-tree meta page is extended with 2 additional parameters: * btm_oldest_btpo_xact – oldest btpo_xact among of deleted pages, * btm_last_cleanup_num_heap_tuples – number of heap tuples dur

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-08 Thread Tomas Vondra
On 03/08/2018 11:46 PM, Peter Geoghegan wrote: > On Thu, Mar 8, 2018 at 6:52 AM, Robert Haas wrote: >>> I removed this code since it was wrong. We might want to add some basic >>> checks for existence of volatile functions in the WHEN or SET clauses. But I >>> agree, it's no different than regul

Re: Testbed for predtest.c ... and some arguable bugs therein

2018-03-08 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 8, 2018 at 12:33 AM, Tom Lane wrote: >> A bit of hacking later, I have the attached. The set of test cases it >> includes at the moment were mostly developed with an eye to getting to >> full code coverage of predtest.c, but we could add more later. What's >> r

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-08 Thread Peter Geoghegan
On Thu, Mar 8, 2018 at 6:52 AM, Robert Haas wrote: >> I removed this code since it was wrong. We might want to add some basic >> checks for existence of volatile functions in the WHEN or SET clauses. But I >> agree, it's no different than regular UPDATEs. So may be not a big deal. > > I just caugh

using worker_spi as pattern

2018-03-08 Thread Jeremy Finzel
Hello - I have compiled and installed the extension worker_spi. I also launched the process via SELECT worker_spi_launch(1); I see this in pg_stat_activity: WITH deleted AS (DELETE FROM schema1.counted WHERE type = 'delta' RETURNING value), total AS (SELECT coalesce(sum(value), 0) as sum FROM del

Re: Implementing SQL ASSERTION

2018-03-08 Thread David Fetter
On Thu, Mar 08, 2018 at 09:11:58PM +, Joe Wildish wrote: > Hi David, > > > > > This patch no longer applies. Any chance of a rebase? > > Of course. I’ll look at it this weekend, Much appreciate it! Best, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 Remember to vote!

Re: Implementing SQL ASSERTION

2018-03-08 Thread Joe Wildish
Hi David, > > This patch no longer applies. Any chance of a rebase? > Of course. I’ll look at it this weekend, Cheers, -Joe

Re: JIT compiling with LLVM v11

2018-03-08 Thread Thomas Munro
On Fri, Mar 9, 2018 at 9:12 AM, Andres Freund wrote: > Or even in core LLVM, which has this nice comment: > > // If user asked for the 'native' CPU, we need to autodetect features. > // This is necessary for x86 where the CPU might not support all the > // features the autodetected CPU name

Re: RFC: Add 'taint' field to pg_control.

2018-03-08 Thread Andres Freund
On 2018-03-07 23:34:37 -0500, Tom Lane wrote: > Craig Ringer writes: > > As I understand it, because we allow multiple Pg instances on a system, we > > identify the small sysv shmem segment we use by the postmaster's pid. If > > you remove the DirLockFile (postmaster.pid) you remove the interlock

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Andres Freund
On 2018-03-08 14:25:59 -0500, Tom Lane wrote: > Andres Freund writes: > > On March 8, 2018 10:46:53 AM PST, Tom Lane wrote: > >> Breaking fundamental invariants like > >> "ctid points to this tuple or its update successor" is going to cause > >> trouble. There's a lot of code that knows that; mo

Re: JIT compiling with LLVM v11

2018-03-08 Thread Andres Freund
On 2018-03-08 11:58:41 -0800, Andres Freund wrote: > I think we can easily fix this by behaving like clang, which uses > llvm::sys::getHostCPUFeatures(HostFeatures) to built the feature list: > > // If -march=native, autodetect the feature list. > if (const Arg *A = Args.getLastArg(clang::driv

Re: JIT compiling with LLVM v11

2018-03-08 Thread Andres Freund
Hi, On 2018-03-09 00:33:03 +1300, Thomas Munro wrote: > On Wed, Mar 7, 2018 at 3:49 PM, Thomas Munro > wrote: > > make check at today's HEAD of your jit branch crashes on my FreeBSD > > box. The first thing to crash is this query from point.sql: > > > > LOG: server process (PID 87060) was termi

Re: parallel append vs. simple UNION ALL

2018-03-08 Thread Robert Haas
On Thu, Mar 1, 2018 at 8:30 AM, Ashutosh Bapat wrote: > The patches look clean. I particularly looked at 0003. > > patch 0001 > +/* > + * Generate partial paths for final_rel, too, if outer query levels might > + * be able to make use of them. > + */ > I am not able to understand t

Re: parallel append vs. simple UNION ALL

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 2:46 AM, Rajkumar Raghuwanshi wrote: > On Thu, Mar 8, 2018 at 12:27 AM, Robert Haas wrote: >> >> New patches attached, fixing all 3 of the issues you reported: > > Thanks. new patches applied cleanly on head and fixing all reported issue. Great. Committed 0001. Are you p

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Tom Lane
Andres Freund writes: > On March 8, 2018 10:46:53 AM PST, Tom Lane wrote: >> Breaking fundamental invariants like >> "ctid points to this tuple or its update successor" is going to cause >> trouble. There's a lot of code that knows that; more than knows the >> details of what's in xmax, I believ

Re: disable SSL compression?

2018-03-08 Thread Claudio Freire
On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut wrote: > It appears that SSL compression is nowadays deprecated as insecure. > Yet, it is still enabled by libpq by default, and there is no way to > disable it in the server. Should we make some changes here? Does > anyone know more about this?

Re: Handling better supported channel binding types for SSL implementations

2018-03-08 Thread Peter Eisentraut
On 1/23/18 21:27, Michael Paquier wrote: > On Tue, Jan 23, 2018 at 12:08:37PM -0500, Peter Eisentraut wrote: >> On 1/22/18 02:29, Michael Paquier wrote: >>> However there is as well the argument that this list's contents are not >>> directly used now, and based on what I saw from the MacOS SSL and

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Andres Freund
On March 8, 2018 10:46:53 AM PST, Tom Lane wrote: >Robert Haas writes: >> On Thu, Mar 8, 2018 at 12:03 PM, Tom Lane wrote: >>> FWIW, I would also vote for (1), especially if the only way to do >(2) >>> is stuff as outright scary as this. I would far rather have (3) >than >>> this, because IMO

Re: Temporary tables prevent autovacuum, leading to XID wraparound

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 1:09 PM, Tom Lane wrote: > We already have autovacuum taking care of that, and as I stated, asking > backends to do it is provably insufficient. The right path is to make > autovacuum cover the cases it's missing today. Well, your counter-proposal of teaching autovacuum to

Re: [HACKERS] GnuTLS support

2018-03-08 Thread Peter Eisentraut
In the thread about Secure Transport we agreed to move the consideration of new SSL libraries to PG12. Here is my current patch, after all the refactorings. The status is that it works fine and could be used. There are two failures in the SSL tests that I cannot explain. The tests are for some

Re: public schema default ACL

2018-03-08 Thread Robert Haas
On Wed, Mar 7, 2018 at 5:11 PM, David G. Johnston wrote: > I still feel like I want to mull this over more but auto-creating schemas > strikes me as being "spooky action at a distance". I don't think that it's a terrible proposal, but I don't see it as fixing the real issue. If we do something e

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Andres Freund
On March 8, 2018 10:46:53 AM PST, Tom Lane wrote: >Robert Haas writes: >> On Thu, Mar 8, 2018 at 12:03 PM, Tom Lane wrote: >>> FWIW, I would also vote for (1), especially if the only way to do >(2) >>> is stuff as outright scary as this. I would far rather have (3) >than >>> this, because IMO

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 8, 2018 at 12:03 PM, Tom Lane wrote: >> FWIW, I would also vote for (1), especially if the only way to do (2) >> is stuff as outright scary as this. I would far rather have (3) than >> this, because IMO, what we are looking at right now is going to make >> the f

disable SSL compression?

2018-03-08 Thread Peter Eisentraut
It appears that SSL compression is nowadays deprecated as insecure. Yet, it is still enabled by libpq by default, and there is no way to disable it in the server. Should we make some changes here? Does anyone know more about this? -- Peter Eisentraut http://www.2ndQuadrant.com/ Pos

Re: Testbed for predtest.c ... and some arguable bugs therein

2018-03-08 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 8, 2018 at 12:33 AM, Tom Lane wrote: >> I'm not sure that that's worth fixing right now. Instead I'm tempted >> to revert the addition of the clause_is_check argument to >> predicate_refuted_by, on the grounds that it's both broken and currently >> unnecessary.

Re: Temporary tables prevent autovacuum, leading to XID wraparound

2018-03-08 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 6, 2018 at 6:39 PM, Tom Lane wrote: >> Now as for the problem originally stated, step 1 alone doesn't fix it, >> and there's reason not to like that change much. Forcing backends to >> clear their temp schemas immediately on connection will slow down >> connecti

Re: FOR EACH ROW triggers on partitioned tables

2018-03-08 Thread Alvaro Herrera
Thomas Munro wrote: > What is this test for? > > +create trigger failed after update on parted_trig > + referencing old table as old_table > + for each statement execute procedure trigger_nothing(); > > It doesn't fail as you apparently expected. Perhaps it was supposed > to be "for each row"

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 12:25 PM, Pavan Deolasee wrote: > I think the question is: isn't there an alternate way to achieve the same > result? One alternate way would be to do what I suggested above i.e. free up > more bits and use one of those. That's certainly possible, but TBH the CTID field see

Re: Temporary tables prevent autovacuum, leading to XID wraparound

2018-03-08 Thread Robert Haas
On Tue, Mar 6, 2018 at 6:39 PM, Tom Lane wrote: > Now as for the problem originally stated, step 1 alone doesn't fix it, > and there's reason not to like that change much. Forcing backends to > clear their temp schemas immediately on connection will slow down > connection times, and for applicati

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Pavan Deolasee
On Thu, Mar 8, 2018 at 10:27 PM, Robert Haas wrote: > > However, there's no such thing as a free lunch. We can't use the CTID > field to point to a CTID in another table because there's no room to > include the identify of the other table in the field. We can't widen > it to make room because t

Re: Testbed for predtest.c ... and some arguable bugs therein

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 12:33 AM, Tom Lane wrote: > A bit of hacking later, I have the attached. The set of test cases it > includes at the moment were mostly developed with an eye to getting to > full code coverage of predtest.c, but we could add more later. What's > really interesting is that i

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 12:03 PM, Tom Lane wrote: > Robert Haas writes: >> Therefore, the only options are (1) ignore the problem, and let a >> cross-partition update look entirely like a delete+insert, (2) try to >> throw some error in the case where this introduces user-visible >> anomalies that

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Tom Lane
Robert Haas writes: > Therefore, the only options are (1) ignore the problem, and let a > cross-partition update look entirely like a delete+insert, (2) try to > throw some error in the case where this introduces user-visible > anomalies that wouldn't be visible otherwise, or (3) revert update > t

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 11:45 AM, Tom Lane wrote: > Prabhat Sahu writes: >> On Wed, Mar 7, 2018 at 7:51 PM, Robert Haas wrote: >>> That looks like the background worker got killed by the OOM killer. How >>> much memory do you have in the machine where this occurred? > >> I have ran the testcase

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 10:07 AM, Tom Lane wrote: > Pavan Deolasee writes: >> I am actually very surprised that 0001-Invalidate-ip_blkid-v5.patch does >> not do anything to deal with the fact that t_ctid may no longer point to >> itself to mark end of the chain. I just can't see how that would wor

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-03-08 Thread Tom Lane
Prabhat Sahu writes: > On Wed, Mar 7, 2018 at 7:51 PM, Robert Haas wrote: >> That looks like the background worker got killed by the OOM killer. How >> much memory do you have in the machine where this occurred? > I have ran the testcase in my local machine with below configurations: > Environm

Re: Server won't start with fallback setting by initdb.

2018-03-08 Thread Tom Lane
Robert Haas writes: > On Wed, Mar 7, 2018 at 6:43 PM, Michael Paquier wrote: >> On Wed, Mar 07, 2018 at 06:39:32PM -0500, Tom Lane wrote: >>> OK, seems like I'm on the short end of that vote. I propose to push the >>> GUC-crosschecking patch I posted yesterday, but not the default-value >>> chan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-03-08 Thread Prabhat Sahu
On Wed, Mar 7, 2018 at 7:51 PM, Robert Haas wrote: > On Wed, Mar 7, 2018 at 8:59 AM, Prabhat Sahu < > prabhat.s...@enterprisedb.com> wrote: >> >> 2018-03-07 19:24:44.263 IST [54400] LOG: background worker "parallel >> worker" (PID 54482) was terminated by signal 9: Killed >> > > That looks like

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Tom Lane
Robert Haas writes: > ... I suppose we could still decide that if we > can't have that, we don't want update tuple routing at all, but I > think that's an overreaction. Between this thread and I am getting the distinct impression that that feature wasn't ready to be committed. I think that reve

Re: unique indexes on partitioned tables

2018-03-08 Thread Alvaro Herrera
Hi, Shinoda, Noriyoshi wrote: > I tried this feature with the latest snapshot. When I executed the > following SQL statement, multiple primary keys were created on the > partition. Is this the intended behavior? Hahah. Is that a serious question? Of course it isn't. I'll fix it: > -- test >

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Tom Lane
Pavan Deolasee writes: > I am actually very surprised that 0001-Invalidate-ip_blkid-v5.patch does > not do anything to deal with the fact that t_ctid may no longer point to > itself to mark end of the chain. I just can't see how that would work. > ... > I am actually worried that we're tinkering w

RE: unique indexes on partitioned tables

2018-03-08 Thread Shinoda, Noriyoshi
Hi. I tried this feature with the latest snapshot. When I executed the following SQL statement, multiple primary keys were created on the partition. Is this the intended behavior? -- test postgres=> CREATE TABLE part1(c1 INT PRIMARY KEY, c2 INT, c3 VARCHAR(10)) PARTITION BY RANGE(c1) ; CREATE

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 7:54 AM, Pavan Deolasee wrote: >> > + /* >> > +* SQL Standard says that WHEN AND conditions must not >> > +* write to the database, so check we haven't written >> > +* any WAL during the test. Very sensible that is, since >> > +

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-08 Thread Jeevan Chalke
On Thu, Mar 8, 2018 at 7:49 PM, Robert Haas wrote: > On Thu, Mar 8, 2018 at 9:15 AM, Jeevan Chalke > wrote: > > I am not sure why we don't set reltarget into the grouped_rel too. > > > > But if we do so like we did in partially_grouped_rel, then it will be lot > > easier for partitionwise aggreg

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 9:15 AM, Jeevan Chalke wrote: > I am not sure why we don't set reltarget into the grouped_rel too. > > But if we do so like we did in partially_grouped_rel, then it will be lot > easier for partitionwise aggregate as then we don't have to pass target to > functions creating

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-08 Thread Jeevan Chalke
On Thu, Mar 8, 2018 at 1:15 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Wed, Mar 7, 2018 at 8:07 PM, Jeevan Chalke > wrote: > Here are some more review comments esp. on > try_partitionwise_grouping() function. BTW name of that function > doesn't go in sync with enable_partit

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 12:34 AM, Pavan Deolasee wrote: > I am actually very surprised that 0001-Invalidate-ip_blkid-v5.patch does not > do anything to deal with the fact that t_ctid may no longer point to itself > to mark end of the chain. I just can't see how that would work. But if it > does, it

Re: [HACKERS] Subscription code improvements

2018-03-08 Thread Masahiko Sawada
On Wed, Mar 7, 2018 at 11:13 PM, Alvaro Herrera wrote: > 0002 looks like a good improvement to me. The existing routine is > messy, and apparently it's so just to save one LockSharedObject plus > cache lookup; IMO it's not worth it. Patched code looks simpler. If > there are cases where having

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-08 Thread Robert Haas
On Thu, Mar 8, 2018 at 2:45 AM, Ashutosh Bapat wrote: > +grouped_rel->part_scheme = input_rel->part_scheme; > +grouped_rel->nparts = nparts; > +grouped_rel->boundinfo = input_rel->boundinfo; > +grouped_rel->part_rels = part_rels; > > You need to set the part_exprs w

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-08 Thread Julian Markwort
I've goofed up now, sorry for failing to attach my updated patch. Am Donnerstag, den 08.03.2018, 14:55 +0100 schrieb Julian Markwort: > Tom Lane wrote on 2018-03-02: > > You need to make your changes in a 1.5--1.6 > > upgrade file. Otherwise there's no clean path for existing > > installations >

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-08 Thread Julian Markwort
Tom Lane wrote on 2018-03-02: > You need to make your changes in a 1.5--1.6 > upgrade file. Otherwise there's no clean path for existing > installations > to upgrade to the new version. I've adressed all the issues that were brought up so far: 1. there is now only an added 1.5--1.6.sql file. 2. t

Re: Failed to request an autovacuum work-item in silence

2018-03-08 Thread Alvaro Herrera
Hi I was thinking that the BRIN code requesting the workitem would print the error message based on the return value. There is no point to returning a boolean indicator if the caller isn't going to do anything with it ... This means you don't need to convert the type to string in autovacuum.c (w

Re: ALTER TABLE ADD COLUMN fast default

2018-03-08 Thread David Rowley
On 8 March 2018 at 18:40, Andrew Dunstan wrote: > select * from t; > fastdef tps = 107.145811 > master tps = 150.207957 > > "select * from t" used to be about a wash, but with this patch it's > got worse. The last two queries were worse and are now better, so > that's a win. How does it compa

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-08 Thread Shubham Barai
On 07-Mar-2018 11:00 PM, "Alvaro Herrera" wrote: I suggest to create a new function GinPredicateLockPage() that checks whether fast update is enabled for the index. The current arrangement looks too repetitive and it seems easy to make a mistake. Stylistically, please keep #include lines ordere

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread amul sul
On Thu, Mar 8, 2018 at 12:31 PM, Amit Kapila wrote: > On Thu, Mar 8, 2018 at 11:04 AM, Pavan Deolasee > wrote: >> [.] > >> But if it >> does, it needs good amount of comments explaining why and most likely >> updating comments at other places where chain following is done. For >> example, how

Re: JIT compiling with LLVM v11

2018-03-08 Thread Thomas Munro
On Wed, Mar 7, 2018 at 3:49 PM, Thomas Munro wrote: > make check at today's HEAD of your jit branch crashes on my FreeBSD > box. The first thing to crash is this query from point.sql: > > LOG: server process (PID 87060) was terminated by signal 4: Illegal > instruction > DETAIL: Failed process

Concurrency bug in UPDATE of partition-key

2018-03-08 Thread Amit Khandekar
(Mail subject changed; original thread : [1]) On 8 March 2018 at 11:57, Amit Khandekar wrote: > On 8 March 2018 at 09:15, Pavan Deolasee wrote: >> >> CREATE TABLE pa_target (key integer, val text) >> PARTITION BY LIST (key); >> CREATE TABLE part1 PARTITION OF pa_target FOR VALUES IN (1); >>

Re: csv format for psql

2018-03-08 Thread Pavel Stehule
2018-03-08 11:17 GMT+01:00 Pavel Stehule : > > > 2018-03-08 11:05 GMT+01:00 Fabien COELHO : > >> >> Hello Daniel, >> >> PFA a v3 patch that implements that, along with regression tests this >>> time. >>> >> >> My 0.02 €: >> >> Patch applies cleanly, compiles, make check ok, doc generation ok. >> >

Re: csv format for psql

2018-03-08 Thread Pavel Stehule
2018-03-08 11:05 GMT+01:00 Fabien COELHO : > > Hello Daniel, > > PFA a v3 patch that implements that, along with regression tests this time. >> > > My 0.02 €: > > Patch applies cleanly, compiles, make check ok, doc generation ok. > > I'm in favor of having a simple psql way to generate a convenien

Re: csv format for psql

2018-03-08 Thread Fabien COELHO
Hello Daniel, PFA a v3 patch that implements that, along with regression tests this time. My 0.02 €: Patch applies cleanly, compiles, make check ok, doc generation ok. I'm in favor of having a simple psql way to generate a convenient and compliant CSV output for export/import. I also thi

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread amul sul
On Thu, Mar 8, 2018 at 3:01 PM, Amit Kapila wrote: > On Thu, Mar 8, 2018 at 12:52 PM, Pavan Deolasee > wrote: >> >> On Thu, Mar 8, 2018 at 12:31 PM, Amit Kapila >> wrote: >>> >>> On Thu, Mar 8, 2018 at 11:04 AM, Pavan Deolasee >>> wrote: >>> > >>> > On Tue, Feb 13, 2018 at 12:41 PM, amul sul w

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-08 Thread Amit Kapila
On Thu, Mar 8, 2018 at 12:52 PM, Pavan Deolasee wrote: > > On Thu, Mar 8, 2018 at 12:31 PM, Amit Kapila > wrote: >> >> On Thu, Mar 8, 2018 at 11:04 AM, Pavan Deolasee >> wrote: >> > >> > On Tue, Feb 13, 2018 at 12:41 PM, amul sul wrote: >> >> >> >> Thanks for the confirmation, updated patch att

Re: [PATCH] btree_gin, add support for uuid, bool, name, bpchar and anyrange types

2018-03-08 Thread Matheus de Oliveira
Hi all. Em 4 de mar de 2018 16:00, "Tomas Vondra" escreveu: 1) I personally am not that sure GIN indexes on ranges are very useful, considering those columns are usually queried for containment (i.e. is this value contained in the range) rather than equality. And we already have gist/spgist opc

Re: Failed to request an autovacuum work-item in silence

2018-03-08 Thread Ildar Musin
Hello, The patch applies and compiles cleanly, tests pass. The code is working as well. I was able to check it by simply creating a BRIN index and filling the table with data forcing the index to request lots of work items: create table test (a serial, b text); create index on test using brin (a)

Re: csv format for psql

2018-03-08 Thread Pavel Stehule
2018-03-08 9:29 GMT+01:00 Fabien COELHO : > > I.e. really generate some csv from the data in just one option, not many. >>> >>> But this is obviously debatable. >>> >> >> I suspect we'll get requests for an all-JSON option, HTML tables, >> etc., assuming we don't have them already. >> > > I would

Re: csv format for psql

2018-03-08 Thread Fabien COELHO
I.e. really generate some csv from the data in just one option, not many. But this is obviously debatable. I suspect we'll get requests for an all-JSON option, HTML tables, etc., assuming we don't have them already. I would definitely be fine with --html (which indeed already exists) & --j

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-08 Thread David Gould
On Wed, 7 Mar 2018 21:39:08 -0800 Jeff Janes wrote: > As for preventing it in the first place, based on your description of your > hardware and operations, I was going to say you need to increase the max > number of autovac workers, but then I remembered you from "Autovacuum slows > down with lar