Re: [HACKERS] proposal: schema variables

2018-03-11 Thread Pavel Stehule
2018-03-12 7:49 GMT+01:00 Pavel Luzanov : > Hi, > > I plan to make usability and feature test review in several days. > > Is there any chances that it will work on replicas? > Such possibility is very helpful in generating reports. > Now, LET command produces an error: > > ERROR: cannot execute L

Re: [HACKERS] proposal: schema variables

2018-03-11 Thread Pavel Luzanov
Hi, I plan to make usability and feature test review in several days. Is there any chances that it will work on replicas? Such possibility is very helpful in generating reports. Now, LET command produces an error: ERROR:  cannot execute LET in a read-only transaction But if we say that variabl

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-11 Thread Andrey Borodin
> 12 марта 2018 г., в 1:54, Alexander Korotkov > написал(а): > > On Wed, Mar 7, 2018 at 8:30 PM, Alvaro Herrera > wrote: > I suggest to create a new function GinPredicateLockPage() that checks > whether fast update is enabled for the index. The current arrangement > looks too repetitive and

Re: Inconsistent behavior in serializable snapshot

2018-03-11 Thread Kuntal Ghosh
On Sun, Mar 11, 2018 at 7:52 PM, Kuntal Ghosh wrote: > Hello hackers, > > While working on serializable transaction isolation, I've noticed some > strange behavior in the first permutation mentioned in > isolation/specs/read-only-anomaly-2.spec file. > > setup > { > CREATE TABLE bank_account (

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-11 Thread amul sul
On Sat, Mar 10, 2018 at 5:25 PM, Amit Kapila wrote: > On Fri, Mar 9, 2018 at 3:18 PM, amul sul wrote: >> On Thu, Mar 8, 2018 at 12:31 PM, Amit Kapila wrote: >>> On Thu, Mar 8, 2018 at 11:04 AM, Pavan Deolasee >>> This is just one example. I am almost certain there are many such cases

WARNING in parallel index creation.

2018-03-11 Thread Jeff Janes
If i run: pgbench -i -s30 And then create the function: CREATE OR REPLACE FUNCTION foobar(text) RETURNS text LANGUAGE plperl IMMUTABLE PARALLEL SAFE STRICT COST 1 AS $function$ return scalar reverse($_[0]); $function$; Then when I create in index, I get a warning: jjanes=# create inde

Re: inserts into partitioned table may cause crash

2018-03-11 Thread Etsuro Fujita
(2018/03/09 20:18), Etsuro Fujita wrote: Here are updated patches for PG10 and HEAD. Other changes: * Add regression tests based on your test cases shown upthread I added a little bit more regression tests and revised comments. Please find attached an updated patch. Best regards, Etsuro Fu

Re: Fixes for missing schema qualifications

2018-03-11 Thread Michael Paquier
On Sat, Mar 10, 2018 at 03:13:09PM -0300, Alvaro Herrera wrote: > ... and substring() ... substring(A from B for C) gets parsed. -- Michael signature.asc Description: PGP signature

Re: Foreign keys and partitioned tables

2018-03-11 Thread Alvaro Herrera
[ Resending an email from yesterday. Something is going very wrong with my outgoing mail provider :-( ] Rebase of the prior code, on top of the improved row triggers posted elsewhere. I added some more tests too, and fixed a couple of small bugs. (This includes the patches I just posted in the

Re: FOR EACH ROW triggers on partitioned tables

2018-03-11 Thread Thomas Munro
On Fri, Mar 9, 2018 at 7:06 AM, Alvaro Herrera wrote: > Thomas Munro wrote: >> +create trigger failed after update on parted_trig >> + referencing old table as old_table >> + for each statement execute procedure trigger_nothing(); >> >> It doesn't fail as you apparently expected. Perhaps it was

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-11 Thread Alvaro Herrera
On 0002: In terms of docs, I think it's better not to have anything user-facing in the README. Consider that users are going to be reading the HTML docs only, and many of them may not have the README available at all. So anything that could be useful to users must be in the XML docs only; keep in

Re: Fixes for missing schema qualifications

2018-03-11 Thread Alvaro Herrera
Noah Misch wrote: > On Fri, Mar 09, 2018 at 04:55:38PM +0900, Michael Paquier wrote: > > --- a/src/backend/catalog/information_schema.sql > > +++ b/src/backend/catalog/information_schema.sql > > @@ -186,7 +186,7 @@ CREATE FUNCTION _pg_interval_type(typid oid, mod int4) > > RETURNS text > > AS

Re: FOR EACH ROW triggers on partitioned tables

2018-03-11 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 3/7/18 20:57, Alvaro Herrera wrote: > > So, unless someone has a brilliant idea on how to construct a column > > mapping from partitioned table to partition, I'm going back to the > > design I was proposing earlier, ie., creating individual pg_trigger rows > > for each

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-11 Thread Michael Paquier
On Sun, Mar 11, 2018 at 09:56:33AM -0500, Justin Pryzby wrote: > Hopefully a set of CHUNKS not JUNKS ? Yes. (laugh) -- Michael signature.asc Description: PGP signature

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-11 Thread Alexander Korotkov
On Wed, Mar 7, 2018 at 8:30 PM, Alvaro Herrera wrote: > I suggest to create a new function GinPredicateLockPage() that checks > whether fast update is enabled for the index. The current arrangement > looks too repetitive and it seems easy to make a mistake. > BTW, should we also skip CheckForSe

Re: Using JIT for VACUUM, COPY, ANALYZE

2018-03-11 Thread Andres Freund
On 2018-03-11 12:38:54 -0700, Andres Freund wrote: > > > On March 11, 2018 12:31:33 PM PDT, Pavel Stehule > wrote: > >Hi > > > >Today, these task can be CPU limited . Do you think, so JIT can be used > >there too? > > Copy definitely, with the others I'm much more doubtful. Don't see anything

Re: Parallel index creation does not properly cleanup after error

2018-03-11 Thread Peter Geoghegan
On Sun, Mar 11, 2018 at 3:22 AM, David Rowley wrote: > Due to the failure during the index build, it appears that the > PG_TRY/PG_CATCH block in reindex_relation() causes the reindex_index() > to abort and jump out to the catch block. Here there's a call to > ResetReindexPending(), which complains

Re: Using JIT for VACUUM, COPY, ANALYZE

2018-03-11 Thread Andres Freund
On March 11, 2018 12:31:33 PM PDT, Pavel Stehule wrote: >Hi > >Today, these task can be CPU limited . Do you think, so JIT can be used >there too? Copy definitely, with the others I'm much more doubtful. Don't see anything around their bottlenecks that could be removed by JITing. Haven't look

Re: Sample values for pg_stat_statements

2018-03-11 Thread Greg Stark
I've often wanted something similar. But I've struggled to come up with a good way to decide which parameters to keep. And as someone mentioned, there's the question of how to deal with very large constants. The other day I was poking around with pg_stat_statements and jsonlog and I thought of ano

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-03-11 Thread Dmitry Dolgov
> On 26 February 2018 at 11:03, Ashutosh Bapat > wrote: > On Fri, Feb 23, 2018 at 7:35 PM, Robert Haas wrote: >> On Fri, Feb 16, 2018 at 12:14 AM, Ashutosh Bapat >> wrote: >>> Appreciate you taking time for review. >>> >>> PFA updated version. >> >> Committed 0001. > > Thanks. > > Here's patchs

Using JIT for VACUUM, COPY, ANALYZE

2018-03-11 Thread Pavel Stehule
Hi Today, these task can be CPU limited . Do you think, so JIT can be used there too? Regards Pavel

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-03-11 Thread Alexander Korotkov
On Sat, Mar 3, 2018 at 2:53 PM, Amit Kapila wrote: > On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro > > If that is indeed a race, could it be fixed by > > calling PredicateLockPageSplit() at the start of _hash_splitbucket() > > instead? > > > > Yes, but I think it would be better if we call this o

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-03-11 Thread Alexander Korotkov
On Fri, Mar 2, 2018 at 6:57 AM, Thomas Munro wrote: > My thought experiments about pseudo-pages and avoiding the split stuff > were not intended to get the patch kicked out. I thought for a while > that hash indexes were a special case and could benefit from > dispensing with those trickier prob

Re: [GSOC 18] Performance Farm Project——Initialization Project

2018-03-11 Thread Dave Page
Hi Maybe I’m missing something (I’ve been offline a lot recently for unavoidable reasons), but the perf farm project already has a Django backend initialised and configured to work with community auth, on community infrastructure. https://git.postgresql.org/gitweb/?p=pgperffarm.git;a=summary On

Re: JIT compiling with LLVM v11

2018-03-11 Thread Andres Freund
On 2018-03-11 13:19:57 -0400, Peter Eisentraut wrote: > On 3/9/18 15:56, Andres Freund wrote: > > I think that's largely that unnecessary trivial queries get JITed and > > optimized, because the stats are entirely completely off. > > Right. I instrumented this a bit, and there are indeed two hand

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-11 Thread Mark Dilger
> On Mar 3, 2018, at 2:40 PM, Tomas Vondra wrote: > > An updated patch version, fixing the breakage caused by fd1a421fe6 > twiddling with pg_proc. Hi Tomas! Reviewing the sgml documentation, I think something like the following should be added: diff --git a/doc/src/sgml/catalogs.sgml b/doc/s

Re: disable SSL compression?

2018-03-11 Thread Tom Lane
Peter Eisentraut writes: > The change in the Debian package I found was to build without zlib at > all. So no amount of turning it back on will help. Whereas the > upstream change was just to make the default to be off. But anyway, > this feature is clearly dying, so we probably shouldn't be tr

Re: Bogus use of canonicalize_qual

2018-03-11 Thread Tom Lane
Dean Rasheed writes: > On 10 March 2018 at 20:21, Tom Lane wrote: >> If we suppose that we only need to fix it in HEAD, the most attractive >> answer is to add a parameter distinguishing WHERE and CHECK arguments >> to canonicalize_qual. > I agree that this looks like the best choice, but it fee

Re: JIT compiling with LLVM v11

2018-03-11 Thread Peter Eisentraut
On 3/9/18 15:42, Peter Eisentraut wrote: > The default of jit_above_cost = 50 seems pretty good. I constructed > a query that cost about 45 where the run time with and without JIT > were about even. This is obviously very limited testing, but it's a > good start. Actually, the default in

Re: JIT compiling with LLVM v11

2018-03-11 Thread Peter Eisentraut
On 3/9/18 15:56, Andres Freund wrote: > On 2018-03-09 15:28:19 -0500, Peter Eisentraut wrote: >> On 3/6/18 15:16, Andres Freund wrote: >>> 2) Don't load the JIT provider until fully needed. Right now >>>jit_compile_expr() will load the jit provider even if not really >>>needed. We should pr

GSOC 2018 proposal

2018-03-11 Thread Charles Cui
Hi Aleksander, I am currently preparing a proposal for pg_thrift project. I noticed that there are several protocols supported by thrift, which ones do we have higher priority? I mean which ones I need to implement during this project? Thanks, Charles.

Re: disable SSL compression?

2018-03-11 Thread Magnus Hagander
On Sun, Mar 11, 2018 at 2:05 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/11/18 04:00, Magnus Hagander wrote: > > I am not talking about the OpenSSL disabling it. It was disabled on most > > *distributions* years ago, long before that commit. Which is why I'm > > still cu

Re: Faster inserts with mostly-monotonically increasing values

2018-03-11 Thread Claudio Freire
On Sun, Mar 11, 2018 at 2:27 AM, Pavan Deolasee wrote: > > > On Sat, Mar 10, 2018 at 12:11 AM, Claudio Freire > wrote: >> >> On Fri, Mar 9, 2018 at 2:54 PM, Pavan Deolasee >> wrote: >> > >> > >> >> > >> > So yes, the benefits of the patch go down with higher number of clients, >> > but >> > it d

[GSOC 18] Performance Farm Project——Initialization Project

2018-03-11 Thread Hongyuan Ma
Hello, mark I initialized a Django project and imported the Django REST Framework. Its github address is: https://github.com/PGPerfFarm/server-code I created some model classes and also wrote scripts in the dbtools folder to import simulation data for development. I am hesitant to use admin or xa

Re: ALTER TABLE ADD COLUMN fast default

2018-03-11 Thread David Rowley
On 9 March 2018 at 02:11, David Rowley wrote: > On 8 March 2018 at 18:40, Andrew Dunstan > wrote: >> select * from t; >> fastdef tps = 107.145811 >> master tps = 150.207957 >> >> "select * from t" used to be about a wash, but with this patch it's >> got worse. The last two queries were worse

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-11 Thread Justin Pryzby
On Sun, Mar 11, 2018 at 11:04:01PM +0900, Michael Paquier wrote: > On Fri, Mar 09, 2018 at 08:22:49AM +, Tsunakawa, Takayuki wrote: > > Thanks for reviewing. All done. > > Thanks. Please be careful with the indentation of the patch. Attached > is a slightly-updated version with a small modi

Inconsistent behavior in serializable snapshot

2018-03-11 Thread Kuntal Ghosh
Hello hackers, While working on serializable transaction isolation, I've noticed some strange behavior in the first permutation mentioned in isolation/specs/read-only-anomaly-2.spec file. setup { CREATE TABLE bank_account (id TEXT PRIMARY KEY, balance DECIMAL NOT NULL); INSERT INTO bank_a

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-11 Thread Michael Paquier
On Fri, Mar 09, 2018 at 08:22:49AM +, Tsunakawa, Takayuki wrote: > Thanks for reviewing. All done. Thanks. Please be careful with the indentation of the patch. Attached is a slightly-updated version with a small modification in remove_target_file to make the code cleaner, a proposal of comm

Re: disable SSL compression?

2018-03-11 Thread Peter Eisentraut
On 3/11/18 04:00, Magnus Hagander wrote: > I am not talking about the OpenSSL disabling it. It was disabled on most > *distributions* years ago, long before that commit. Which is why I'm > still curious as to what platform you actually got it enabled by default > on... Homebrew package > So f

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-11 Thread Amit Kapila
On Thu, Feb 15, 2018 at 4:18 PM, Ashutosh Bapat wrote: > > After recent commits, the patch doesn't get applied cleanly, so rebased it and along the way addressed the comments raised by you. > Here are some comments on the patch. > > +/* > + * Except for the topmos

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-03-11 Thread Alexander Korotkov
On Sat, Mar 10, 2018 at 5:49 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Fri, Mar 9, 2018 at 9:40 PM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada >> wrote: >> >>> Attached an updated patch >>> >> fixed these iss

Re: Parallel Aggregates for string_agg and array_agg

2018-03-11 Thread Tomas Vondra
On 03/11/2018 07:31 AM, David Rowley wrote: > On 11 March 2018 at 12:11, Tomas Vondra wrote: >> On 03/05/2018 04:51 AM, David Rowley wrote: >>> On 5 March 2018 at 04:54, Tomas Vondra wrote: >>> Consider the following slightly backward-looking case; >>> >>> select string_agg(',', x::text) from g

Parallel index creation does not properly cleanup after error

2018-03-11 Thread David Rowley
Hi, I've just stumbled on a bug in the parallel reindexing code. Example: -- force parallel index creation set parallel_tuple_cost = 0; set parallel_setup_cost = 0; set min_parallel_table_scan_size = '0MB'; set min_parallel_index_scan_size = '0kB'; -- example (from the regression tests) CREATE

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-11 Thread Andrey Borodin
> 10 марта 2018 г., в 19:20, Michail Nikolaev > написал(а): > > > Also, I think that this check could be removed from loop. We do not expect > > that it's state will change during execution, do we? > > Removed. Sorry, seems like I've incorrectly expressed what I wanted to say. I mean this

Re: MCV lists for highly skewed distributions

2018-03-11 Thread Dean Rasheed
On 6 March 2018 at 16:48, Dean Rasheed wrote: > On 6 March 2018 at 08:51, John Naylor wrote: >> On 3/5/18, Dean Rasheed wrote: >>> Attached is an updated patch. >> Nice. The results look good. > > Thanks for the review. > So I was about ready to commit this, but decided to do more testing, beca

Re: PATCH: Unlogged tables re-initialization tests

2018-03-11 Thread Michael Paquier
On Fri, Mar 09, 2018 at 05:23:48PM -0500, Peter Eisentraut wrote: > This seems like a useful test. > > On 3/5/18 12:35, David Steele wrote: > > +mkdir($tablespaceDir) > > + or die "unable to mkdir \"$tablespaceDir\""; > > Use BAIL_OUT instead of die in tests. Would it be better to make this pr

Re: Bogus use of canonicalize_qual

2018-03-11 Thread Dean Rasheed
On 10 March 2018 at 20:21, Tom Lane wrote: > I wrote: >> Whilst fooling about with predtest.c, I noticed a rather embarrassing >> error. Consider the following, rather silly, CHECK constraint: >> ... >> So, what to do? We have a few choices, none ideal: > > I'd been assuming that we need to back

Re: initdb help message about WAL segment size

2018-03-11 Thread Michael Paquier
On Sun, Mar 11, 2018 at 08:23:39AM +0100, Magnus Hagander wrote: > I propose the attached patch, aligning the help message with the docs. Any > reason not to? +1 for doing what you are suggesting. -- Michael signature.asc Description: PGP signature

Re: disable SSL compression?

2018-03-11 Thread Magnus Hagander
Sun, Mar 11, 2018 at 12:36 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/9/18 09:06, Magnus Hagander wrote: > > What platform does that actually work out of the box on? I have > > customers who actively want to use it (for compression, not security -- > > replication acro