Re: [HACKERS] path toward faster partition pruning

2018-02-22 Thread David Rowley
On 21 February 2018 at 23:44, Amit Langote wrote: > Please find attached updated patches. Thanks for updating the code. The question I have now is around NULL handling in partkey_datum_from_expr(). I've not managed to find a way to get a NULL Const in there as it seems all the clauses I try get

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread Amit Langote
Hi David. On 2018/02/21 18:06, David Rowley wrote: > Other things I don't particularly like about the current patch are how > I had to construct the partition key expressions in > set_valid_runtime_subplans_recurse(). This pretty much uses code > borrowed from set_baserel_partition_key_exprs(). On

Re: [HACKERS] path toward faster partition pruning

2018-02-22 Thread Amit Langote
On 2018/02/22 17:41, David Rowley wrote: > On 21 February 2018 at 23:44, Amit Langote > wrote: >> Please find attached updated patches. > > Thanks for updating the code. > > The question I have now is around NULL handling in > partkey_datum_from_expr(). I've not managed to find a way to get a >

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
Re-sending this one with proper formatting. Apologies for the horrible gmail-screws-up-the-text-part of the last one! No change to patch or text, just the formatting. //Magnus Once more, here is an attempt to solve the problem of on-line enabling of checksums that me and Daniel have been hack

Re: [HACKERS] path toward faster partition pruning

2018-02-22 Thread David Rowley
On 22 February 2018 at 22:48, Amit Langote wrote: >> I'm having to add some NULL handling there for the run-time pruning >> patch but wondered if it was also required for your patch. > > Hmm, not sure why. Can you explain a bit more? hmm, yeah, but perhaps we should be discussing on the other th

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Claudio Freire
On Wed, Feb 21, 2018 at 11:21 PM, Tomas Vondra wrote: > On 02/21/2018 02:10 AM, Peter Geoghegan wrote: >> On Tue, Feb 20, 2018 at 3:54 PM, Tomas Vondra >> wrote: I suspect that it could make sense to use a Bloom filter to summarize the entire inner side of the join all at once, even >>>

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-02-22 Thread Etsuro Fujita
(2018/02/22 11:52), Amit Langote wrote: On 2018/02/21 20:54, Etsuro Fujita wrote: void BeginForeignRouting(ModifyTableState *mtstate, ResultRelInfo *resultRelInfo, int partition_index); Prepare for a tuple-routing operation on a foreign table. This is

Incorrect grammar

2018-02-22 Thread Etsuro Fujita
Here is a tiny patch to fix $SUBJECT in a comment in execPartition.c. Best regards, Etsuro Fujita diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 4048c3e..882b538 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartiti

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread David Rowley
On 22 February 2018 at 22:31, Amit Langote wrote: > Some comments: Hi Amit, Thanks for looking at this. I'll work through your comments and produce a patch sometime in the near future. One problem that I'm facing now is down to the way I'm gathering the ParamIds that match the partkeys. As you'

PlaceHolderVars in pushed down child-join cause error

2018-02-22 Thread Ashutosh Bapat
Hi, postgres_fdw isn't expected to push down joins with placeholder vars. But the check for that in foreign_join_ok() only considers joinrel->relids. For a child-join relids contains the child relids but PlaceHolderInfo refers to the top parent's relids. Hence postgres_fdw tries to push down a chil

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
On Fri, Jan 26, 2018 at 4:24 AM, Robert Haas wrote: > I took a look at this today and thought it might be OK to commit, Thank you for looking at this! > modulo a few minor issues: (1) you didn't document the new tranche and Fixed. > (2) I prefer to avoid if (blah) { Assert(thing) } in favor of

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
On Fri, Feb 23, 2018 at 1:54 AM, Thomas Munro wrote: > The attached is a draft patch only, needing some testing and polish. > Brickbats, better ideas? Note, that version is broken for multiple Gather nodes, but that's fixable. Comments on the general idea welcome. -- Thomas Munro http://www.en

Re: Drop --disable-floatN-byval configure options?

2018-02-22 Thread Peter Eisentraut
On 2/21/18 12:17, Tom Lane wrote: > I don't actually envision changing the C code much at all; we might want > to resurrect the old code at some point. I just want to reduce the number > of supported configurations. Yeah, it can exist like EXEC_BACKEND, as a way to manually simulate a different a

Re: Online enabling of checksums

2018-02-22 Thread Peter Eisentraut
On 2/21/18 15:53, Magnus Hagander wrote: > *Two new functions are added, pg_enable_data_checksums() and > pg_disable_data_checksums(). The disable one is easy -- it just changes > to disable. The enable one will change the state to inprogress, and then > start a background worker (the “checksumhelp

phasing out pg_pltemplate?

2018-02-22 Thread Peter Eisentraut
What is the plan for pg_pltemplate? Is there a roadmap to get rid of it? (It's not currently blocking anything for me. I'm just wondering.) -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: pgsql: Charge cpu_tuple_cost * 0.5 for Append and MergeAppend nodes.

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 12:35 AM, Tom Lane wrote: > Robert Haas writes: >> Charge cpu_tuple_cost * 0.5 for Append and MergeAppend nodes. > > This seems to have produced some plan instability in the buildfarm. I was worried about that. Looking at it now. I wish we had some way to figure out whe

check error messages in SSL tests

2018-02-22 Thread Peter Eisentraut
I noticed that a couple of test cases in the SSL tests fail to connect not for the reason that the tests think they should. Here is a patch to augment the test setup so that a test for connection rejection also checks that we get the expected error message. -- Peter Eisentraut http:

Re: pgsql: Charge cpu_tuple_cost * 0.5 for Append and MergeAppend nodes.

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 8:31 AM, Robert Haas wrote: > On Thu, Feb 22, 2018 at 12:35 AM, Tom Lane wrote: >> Robert Haas writes: >>> Charge cpu_tuple_cost * 0.5 for Append and MergeAppend nodes. >> >> This seems to have produced some plan instability in the buildfarm. > > I was worried about that.

Re: [doc fix] Correct calculation of vm.nr_hugepages

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 3:28 PM, Justin Pryzby wrote: > On Wed, Feb 21, 2018 at 03:14:57PM -0500, Robert Haas wrote: >> On Mon, Feb 19, 2018 at 9:43 PM, Tsunakawa, Takayuki >> wrote: >> > Thanks, I'd like to take this. >> >> Why are these values so large? The example in the documentation shows >

Re: Possible performance regression in version 10.1 with pgbench read-write tests.

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 10:03 PM, Mithun Cy wrote: > seeing futex in the call stack andres suggested that following commit could > be the reason for regression > > commit ecb0d20a9d2e09b7112d3b192047f711f9ff7e59 > Author: Tom Lane > Date: 2016-10-09 18:03:45 -0400 > > Use unnamed POSIX sema

Re: pgsql: Avoid valgrind complaint about write() of uninitalized bytes.

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 3:18 PM, Peter Geoghegan wrote: > On Wed, Feb 21, 2018 at 10:55 AM, Peter Geoghegan wrote: >> Sure, but it looks like it has the exact same underlying cause to the >> LogicalTapeFreeze() issue. It shouldn't be very hard to write an >> equivalent patch for LogicalTapeRewind

Re: PlaceHolderVars in pushed down child-join cause error

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 7:41 AM, Ashutosh Bapat wrote: > postgres_fdw isn't expected to push down joins with placeholder vars. > But the check for that in foreign_join_ok() only considers > joinrel->relids. For a child-join relids contains the child relids but > PlaceHolderInfo refers to the top p

Re: Incorrect grammar

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 6:54 AM, Etsuro Fujita wrote: > Here is a tiny patch to fix $SUBJECT in a comment in execPartition.c. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread Jesper Pedersen
Hi David, On 02/21/2018 04:06 AM, David Rowley wrote: I've attached v11 of the patch. Are UPDATE and DELETE suppose to be supported ? With -- test.sql -- CREATE TABLE test (a integer NOT NULL, b integer) PARTITION BY HASH(a); CREATE TABLE test_p00 PARTITION OF test FOR VALUES WITH (MODULUS

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread David Rowley
On 23 February 2018 at 01:15, David Rowley wrote: > One problem that I'm facing now is down to the way I'm gathering the > ParamIds that match the partkeys. As you'll see from the patch I've > added a 'paramids' field to PartitionPruneContext and I'm populating > this when the clauses are being pr

Re: phasing out pg_pltemplate?

2018-02-22 Thread Tom Lane
Peter Eisentraut writes: > What is the plan for pg_pltemplate? Is there a roadmap to get rid of > it? (It's not currently blocking anything for me. I'm just wondering.) I think it's just waiting for someone to put in the effort to make it unnecessary. It seems like the extension mechanism cou

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Tomas Vondra
On 02/22/2018 12:44 PM, Claudio Freire wrote: > On Wed, Feb 21, 2018 at 11:21 PM, Tomas Vondra > wrote: >> On 02/21/2018 02:10 AM, Peter Geoghegan wrote: >>> ... >>> I misunderstood. I would probably do something like double or triple >>> the original rows estimate instead, though. The estimate

Re: Online enabling of checksums

2018-02-22 Thread Andrey Borodin
Hello, Magnus, Peter! I'm excited that this feature emerged, thanks for the patch. Hope it will help to fix some mistakes made during initdb long time ago... > 22 февр. 2018 г., в 18:22, Peter Eisentraut > написал(а): > > On 2/21/18 15:53, Magnus Hagander wrote: >> *Two new functions are adde

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Robert Haas
On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote wrote: >> Attached is an updated version for that. > > Thanks for updating the patch. Committed with a few changes. The big one was that I got rid of the local variable is_update in ExecSetupPartitionTupleRouting. That saved a level of indentation o

[PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2018-02-22 Thread Nikolay Shaplov
This is part or my bigger patch https://www.postgresql.org/message-id/flat/2146419.veIEZdk4E4@x200m#2146419.veIEZdk4E4@x200m we've decided to commit by smaller parts. Now in postgres an StdRdOptions structure is used as binary represenations of reloptions for heap, toast, and some indexes. It h

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote > wrote: > >> Attached is an updated version for that. > > > > Thanks for updating the patch. > > Committed with a few changes. I propose to tweak a few comments to PartitionTupleRouting, as attached. -- Álvaro Herrera

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 11:53 AM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote >> wrote: >> >> Attached is an updated version for that. >> > >> > Thanks for updating the patch. >> >> Committed with a few changes. > > I propose to tweak a few comments

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro wrote:> > The best solution I have come up with so far is to add a reference > count to SERIALIZABLEXACT. I toyed with putting the refcount into the > DSM instead, but then I ran into problems making that work when you > have a query with multiple Gat

Translations contributions urgently needed

2018-02-22 Thread Thom Brown
Hi, I have found that Japanese language support for the database server has been dropped for 10. This is because it fell below the 80% of strings translated requirement, so it was shipped without Japanese. This isn't true of all components, but it seems quite alarming that we've pushed out Postgr

Re: Translations contributions urgently needed

2018-02-22 Thread Pavel Stehule
2018-02-22 18:20 GMT+01:00 Thom Brown : > Hi, > > I have found that Japanese language support for the database server > has been dropped for 10. This is because it fell below the 80% of > strings translated requirement, so it was shipped without Japanese. > This isn't true of all components, but

Re: Translations contributions urgently needed

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 6:20 PM, Thom Brown wrote: > Hi, > > I have found that Japanese language support for the database server > has been dropped for 10. This is because it fell below the 80% of > strings translated requirement, so it was shipped without Japanese. > This isn't true of all comp

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2018-02-22 Thread Nikolay Shaplov
В письме от 3 сентября 2017 11:45:43 пользователь Alvaro Herrera написал: > I think we should split this in at least two commits, I've added a third part of this patch to commitfest: https://www.postgresql.org/message-id/flat/2083183.Rn7qOxG4Ov@x200m#2083183.Rn7qOxG4Ov@x200m To finally commit the

Re: Translations contributions urgently needed

2018-02-22 Thread Thom Brown
On 22 February 2018 at 17:24, Magnus Hagander wrote: > On Thu, Feb 22, 2018 at 6:20 PM, Thom Brown wrote: >> >> Hi, >> >> I have found that Japanese language support for the database server >> has been dropped for 10. This is because it fell below the 80% of >> strings translated requirement, so

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 4:47 PM, Andrey Borodin wrote: > Hello, Magnus, Peter! > > I'm excited that this feature emerged, thanks for the patch. Hope it will > help to fix some mistakes made during initdb long time ago... > > 22 февр. 2018 г., в 18:22, Peter Eisentraut com> написал(а): > > On 2/2

Re: Translations contributions urgently needed

2018-02-22 Thread Ioseph Kim
Hi, I’m translating to Korean since 10 years over. I think this issue should be left to the local community. It may be omitted in some versions, or added in some versions. because, it depends on volunteering. Let It Be! Regards Ioseph. On Thu, Feb 22, 2018 at 05:29:11PM +, Thom Brown wrote:

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Tom Lane
Alexander Kuzmenkov writes: > explain select * from j1_tbl full join (select * from j2_tbl order by > j2_tbl.i desc, j2_tbl.k) j2_tbl on j1_tbl.i = j2_tbl.i and j1_tbl.i = > j2_tbl.k; > ERROR: left and right pathkeys do not match in mergejoin Nice example. There are several places that we cou

Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
In working on the checksumhelper patch, we came across wanting a background worker to be allowed to bypass datallowconn for a database. Right now we didn't take care of that, and just said "you have to ALTER TABLE" first. Specifically for this usecase that is OK, but not paticularly user friendly.

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Magnus Hagander writes: > Attached is a patch that adds new Override versions of the functions to > connect to a database from a background worker. > Another option would be to just add the parameter directly to the regular > connection function, and not create separate functions. But that would

Re: SHA-2 functions

2018-02-22 Thread Peter Eisentraut
On 2/22/18 01:05, Michael Paquier wrote: > On Wed, Feb 21, 2018 at 03:45:17PM -0500, Peter Eisentraut wrote: >> On 2/20/18 23:04, Michael Paquier wrote: >>> I think that crypto_hash.c or hash_crypt.c would be adapted as well. >>> crypt.c is too much generic, so including both concepts in the name i

Re: [HACKERS] PoC: full merge join on comparison clause

2018-02-22 Thread Alexander Kuzmenkov
Here are some updates on this patch. I split it into two parts. The preparatory part contains some mechanical changes to prepare for the main part. Most importantly, a new field is added, `RestrictInfo.is_mj_equality`. It is a marker of mergejoinable equality clauses, and `RestrictInfo.mergeop

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 7:24 PM, Tom Lane wrote: > Magnus Hagander writes: > > Attached is a patch that adds new Override versions of the functions to > > connect to a database from a background worker. > > > Another option would be to just add the parameter directly to the regular > > connectio

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 19:01:35 +0100, Magnus Hagander wrote: > In working on the checksumhelper patch, we came across wanting a background > worker to be allowed to bypass datallowconn for a database. Right now we > didn't take care of that, and just said "you have to ALTER TABLE" first. I suspect yo

Re: Allow workers to override datallowconn

2018-02-22 Thread Simon Riggs
On 22 February 2018 at 18:24, Tom Lane wrote: >> Are there any other caveats in doing that this actually makes it dangerous >> to just allow bypassing it for extensions? > > Don't think so; we autovacuum such DBs anyway don't we? Yeh, there is already precedent that should mean it is easy/defaul

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:17 PM, Andres Freund wrote: > Hi, > > On 2018-02-22 19:01:35 +0100, Magnus Hagander wrote: > > In working on the checksumhelper patch, we came across wanting a > background > > worker to be allowed to bypass datallowconn for a database. Right now we > > didn't take care

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
On 2018-02-22 08:22:48 -0500, Peter Eisentraut wrote: > On 2/21/18 15:53, Magnus Hagander wrote: > > *Two new functions are added, pg_enable_data_checksums() and > > pg_disable_data_checksums(). The disable one is easy -- it just changes > > to disable. The enable one will change the state to inpro

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 20:24:03 +0100, Magnus Hagander wrote: > In a background worker you can just set the parameter using > SetConfigOption(), no? That seems a lot easier than turning things in to a > kv pair and back... Sure, but, it doesn't seem bad to offer the option to only allow this for code

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:26 PM, Andres Freund wrote: > Hi, > > On 2018-02-22 20:24:03 +0100, Magnus Hagander wrote: > > In a background worker you can just set the parameter using > > SetConfigOption(), no? That seems a lot easier than turning things in to > a > > kv pair and back... > > Sure, b

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:24 PM, Andres Freund wrote: > On 2018-02-22 08:22:48 -0500, Peter Eisentraut wrote: > > On 2/21/18 15:53, Magnus Hagander wrote: > > > *Two new functions are added, pg_enable_data_checksums() and > > > pg_disable_data_checksums(). The disable one is easy -- it just chang

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Claudio Freire
On Thu, Feb 22, 2018 at 12:45 PM, Tomas Vondra wrote: > > > On 02/22/2018 12:44 PM, Claudio Freire wrote: >> Let me reiterate, you can avoid both issues with scalable bloom filters[1]. >> > > I'm afraid it's not as straight-forward as "Use scalable bloom filters!" > > This is not merely a question

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 20:30:52 +0100, Magnus Hagander wrote: > On Thu, Feb 22, 2018 at 8:24 PM, Andres Freund wrote: > > I suspect I'm going to get some grief for this, but I think the time has > > come to bite the bullet and support changing databases in the same > > process... > > > > Hey, I can't

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:41 PM, Andres Freund wrote: > > On 2018-02-22 20:30:52 +0100, Magnus Hagander wrote: > > On Thu, Feb 22, 2018 at 8:24 PM, Andres Freund > wrote: > > > I suspect I'm going to get some grief for this, but I think the time > has > > > come to bite the bullet and support ch

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
On 2018-02-22 20:30:02 +0100, Magnus Hagander wrote: > Complexity for the bgw usecase. It now has to construct a key/value pair > with proper escaping (well, for this one flag it would be easy, but if we > do that wouldn't we also support the other config params? Were you thinking > we'd have *just

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Tom Lane
I wrote: > The third possibility is to decide that create_mergejoin_plan is being > overly paranoid and it's okay to extract merge details from a "redundant" > path key even though it specifies the opposite sort order from what the > current merge clause seems to need. This is scary at first glanc

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
On February 22, 2018 11:44:17 AM PST, Magnus Hagander wrote: >On Thu, Feb 22, 2018 at 8:41 PM, Andres Freund >wrote: >In this particular case that would at least phase 1 simplify it because >we'd only need one process instead of worker/launcher. However, if we'd >ever want to parallellize it -

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Magnus Hagander writes: > On Thu, Feb 22, 2018 at 8:26 PM, Andres Freund wrote: >> What's the argument against? > Complexity for the bgw usecase. They'd be completely different implementations and code paths, no? For pg_upgrade to use such a thing it'd need to be a connection parameter of some

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Andres Freund writes: > The more important part I think is that we solve it via a GUC that can > be used outside of bgworkers. Are you proposing an "ignore_datallowconn" GUC? That's a remarkably bad idea. We don't have infrastructure that would allow it to be set at an appropriate scope. I can

Re: Online enabling of checksums

2018-02-22 Thread Peter Eisentraut
On 2/22/18 12:38, Magnus Hagander wrote: > I'm not entirely sure which the others ones are. Auto-Vacuum obviously > is one, which doesn't use the worker infrastructure. But I'm not sure > which the others are referring to?  autovacuum, subscription workers, auto prewarm -- Peter Eisentraut

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Tomas Vondra
On 02/22/2018 08:33 PM, Claudio Freire wrote: > On Thu, Feb 22, 2018 at 12:45 PM, Tomas Vondra > wrote: >> >> >> On 02/22/2018 12:44 PM, Claudio Freire wrote: >>> ... >>> >>> An HLL can be used to estimate set size, the paper makes no >>> mention of it, probably assuming only distinct items are ad

Re: Allow workers to override datallowconn

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 9:09 PM, Tom Lane wrote: > Andres Freund writes: > > The more important part I think is that we solve it via a GUC that can > > be used outside of bgworkers. > > Are you proposing an "ignore_datallowconn" GUC? That's a remarkably > bad idea. We don't have infrastructure

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 8:52 PM, Andres Freund wrote: > > > On February 22, 2018 11:44:17 AM PST, Magnus Hagander > wrote: > >On Thu, Feb 22, 2018 at 8:41 PM, Andres Freund > >wrote: > >In this particular case that would at least phase 1 simplify it because > >we'd only need one process instead

Re: Online enabling of checksums

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 21:16:02 +0100, Magnus Hagander wrote: > You could do that, but then you've moving the complexity to managing that > list in shared memory instead. Maybe I'm missing something, but how are you going to get quick parallel processing if you don't have a shmem piece? You can't assi

Re: Allow workers to override datallowconn

2018-02-22 Thread Tom Lane
Magnus Hagander writes: > I hacked up an attempt to do this. It does seem to work in the very simple > case, but it does requiring changing the order in InitPostgres() to load > the startup packet before validating those. I doubt that's safe. It requires, to name just one thing, an assumption th

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
On 2018-02-22 15:24:50 -0500, Tom Lane wrote: > Magnus Hagander writes: > > I hacked up an attempt to do this. It does seem to work in the very simple > > case, but it does requiring changing the order in InitPostgres() to load > > the startup packet before validating those. > > I doubt that's sa

Re: Online enabling of checksums

2018-02-22 Thread Magnus Hagander
On Thu, Feb 22, 2018 at 9:23 PM, Andres Freund wrote: > Hi, > > On 2018-02-22 21:16:02 +0100, Magnus Hagander wrote: > > You could do that, but then you've moving the complexity to managing that > > list in shared memory instead. > > Maybe I'm missing something, but how are you going to get quick

Re: Allow workers to override datallowconn

2018-02-22 Thread Andres Freund
Hi, On 2018-02-22 15:24:50 -0500, Tom Lane wrote: > You could possibly make it work with more aggressive refactoring, but > I remain of the opinion that this is a fundamentally bad idea anyhow. > A GUC of this kind is just ripe for abuse, and I don't think it's solving > any problem we really need

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Claudio Freire
On Thu, Feb 22, 2018 at 5:11 PM, Tomas Vondra wrote: > On 02/22/2018 08:33 PM, Claudio Freire wrote: >> That's kinda slow to do per-item. I tried to "count" distinct items by >> checking the BF before adding (don't add redundantly), but that's less >> precise than a HLL in my experience. > > But y

Re: Add PGDLLIMPORT to enable_hashagg

2018-02-22 Thread Andres Freund
On 2018-02-21 11:41:31 -0800, Brian Cloutier wrote: > On Wed, Feb 21, 2018 at 10:14 AM, Andres Freund wrote: > > > Could you take the relevant commit, backport it to the > > relevant branches, resolve conflicts, make possibly appropriate > > adaptions, and post? > > > > The original commit touch

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Tomas Vondra
On 02/22/2018 09:52 PM, Claudio Freire wrote: > On Thu, Feb 22, 2018 at 5:11 PM, Tomas Vondra > wrote: >> On 02/22/2018 08:33 PM, Claudio Freire wrote: >>> That's kinda slow to do per-item. I tried to "count" distinct items by >>> checking the BF before adding (don't add redundantly), but that's

Re: [HACKERS] Partition-wise aggregation/grouping

2018-02-22 Thread Robert Haas
On Thu, Feb 8, 2018 at 8:05 AM, Jeevan Chalke wrote: > In this attached version, I have rebased my changes over new design of > partially_grouped_rel. The preparatory changes of adding > partially_grouped_rel are in 0001. I spent today hacking in 0001; results attached. The big change from your

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-22 Thread Peter Geoghegan
On Thu, Feb 22, 2018 at 1:14 PM, Tomas Vondra wrote: > OK, thanks for reminding me about SBF and for the discussion. > > At this point I'll probably focus on the other parts though - > determining selectivity of the join, etc. Which I think is crucial, and > we need to get that right even for accu

Re: file cloning in pg_upgrade and CREATE DATABASE

2018-02-22 Thread Peter Eisentraut
On 2/21/18 18:57, Tomas Vondra wrote: > Nice improvement, of course. How does that affect performance on the > cloned database? If I understand this correctly, it essentially enables > CoW on the files, so what's the overhead on that? It'd be unfortunate to > speed up CREATE DATABASE only to get de

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> I have found that Japanese language support for the database server > has been dropped for 10. This is because it fell below the 80% of > strings translated requirement, so it was shipped without Japanese. Was that mentioned in the 10.0 release note? I didn't know that. Best regards, -- Tatsuo

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Alexander Kuzmenkov
El 22/02/18 a las 21:03, Tom Lane escribió: The third possibility is to decide that create_mergejoin_plan is being overly paranoid and it's okay to extract merge details from a "redundant" path key even though it specifies the opposite sort order from what the current merge clause seems to need.

Re: Typo in predicate.c comment

2018-02-22 Thread Robert Haas
On Wed, Feb 21, 2018 at 3:10 PM, Thomas Munro wrote: > Here's a tiny patch to fix a typo. I have pushed a tiny commit containing it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] Pluggable storage

2018-02-22 Thread Robert Haas
On Fri, Feb 16, 2018 at 5:56 AM, Alexander Korotkov wrote: > BTW, EnterpriseDB announces zheap table access method (heap with undo log) > [2]. I think this is great, and I'm looking forward for publishing zheap in > mailing lists. But I'm concerning about its compatibility with pluggable > table

Re: Translations contributions urgently needed

2018-02-22 Thread Robert Haas
On Thu, Feb 22, 2018 at 5:54 PM, Tatsuo Ishii wrote: >> I have found that Japanese language support for the database server >> has been dropped for 10. This is because it fell below the 80% of >> strings translated requirement, so it was shipped without Japanese. > > Was that mentioned in the 10.

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Alvaro Herrera
Alvaro Herrera wrote: > Another option is to rethink this feature from the ground up: instead of > cloning catalog rows for each children, maybe we should have the trigger > lookup code, when running DML on the child relation (the partition), > obtain trigger entries not only for the child relatio

Re: ERROR: left and right pathkeys do not match in mergejoin

2018-02-22 Thread Tom Lane
Alexander Kuzmenkov writes: >> The third possibility is to decide that create_mergejoin_plan is being >> overly paranoid and it's okay to extract merge details from a "redundant" >> path key even though it specifies the opposite sort order from what the >> current merge clause seems to need. > Th

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
>> Was that mentioned in the 10.0 release note? I didn't know that. > > I don't think we ever mention translations that are added or dropped > in minor release notes. I think it would better for users to be noticed that in the release notes because the message translations are apparently visible

Re: PostgreSQL crashes with SIGSEGV

2018-02-22 Thread Peter Geoghegan
On Mon, Feb 12, 2018 at 6:15 PM, Peter Geoghegan wrote: > * For 9.5 and 9.6, the approach taken in bugfix commit d8589946d > should be taken even further -- we should always copy. Moreover, we > should always copy within tuplesort_getdatum(), for the same reasons. > > * For 9.5, 9.6, 10, and maste

Re: Translations contributions urgently needed

2018-02-22 Thread Kyotaro HORIGUCHI
# my clone repository has been corrupted...sigh. At Thu, 22 Feb 2018 18:32:15 -0500, Robert Haas wrote in > On Thu, Feb 22, 2018 at 5:54 PM, Tatsuo Ishii wrote: > >> I have found that Japanese language support for the database server > >> has been dropped for 10. This is because it fell below

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Amit Langote
On 2018/02/23 1:53, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote >> wrote: Attached is an updated version for that. >>> >>> Thanks for updating the patch. >> >> Committed with a few changes. > > I propose to tweak a few comments to PartitionTupl

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> I also noticed the same thing few days ago, by the fact that RPM > release of PG10 doesn't contain psql translation. > > @10 > ja | pg_basebackup,pg_resetxlog,plpython,pltcl,postgres,psql > @9.4> ja | pg_basebackup,pg_resetxlog > > I'm not sure how the translations are maintained

Re: non-bulk inserts and tuple routing

2018-02-22 Thread Amit Langote
On 2018/02/23 1:10, Robert Haas wrote: > On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote > wrote: >>> Attached is an updated version for that. >> >> Thanks for updating the patch. > > Committed with a few changes. The big one was that I got rid of the > local variable is_update in ExecSetupPartiti

Re: Translations contributions urgently needed

2018-02-22 Thread Craig Ringer
On 23 February 2018 at 08:51, Tatsuo Ishii wrote: > > I also noticed the same thing few days ago, by the fact that RPM > > release of PG10 doesn't contain psql translation. > > > > @10 > ja | pg_basebackup,pg_resetxlog, > plpython,pltcl,postgres,psql > > @9.4> ja | pg_basebackup,pg_

Re: [HACKERS] path toward faster partition pruning

2018-02-22 Thread Amit Langote
On 2018/02/22 20:28, David Rowley wrote: > On 22 February 2018 at 22:48, Amit Langote > wrote: >>> I'm having to add some NULL handling there for the run-time pruning >>> patch but wondered if it was also required for your patch. >> >> Hmm, not sure why. Can you explain a bit more? > > hmm, yeah

Re: Translations contributions urgently needed

2018-02-22 Thread Tatsuo Ishii
> I was wondering a bit about how translations are maintained. Experience > with another highly internationalised project years ago showed that many > people who are willing to volunteer as translators are NOT willing to > interact with git (or at the time, svn) and the other tools many of us take

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Amit Langote
On 2018/02/23 8:52, Alvaro Herrera wrote: > We could mitigate the performance loss to some extent by adding more to > RelationData. For example, a "is_partition" boolean would help: skip > searching pg_inherits for a relation that is not a partition. Unless I'm missing something, doesn't rd_rel->r

Re: [HACKERS] Runtime Partition Pruning

2018-02-22 Thread Amit Langote
Hi David. On 2018/02/23 0:11, David Rowley wrote: > On 23 February 2018 at 01:15, David Rowley > wrote: >> One problem that I'm facing now is down to the way I'm gathering the >> ParamIds that match the partkeys. As you'll see from the patch I've >> added a 'paramids' field to PartitionPruneCont

Re: Incorrect grammar

2018-02-22 Thread Etsuro Fujita
(2018/02/23 0:09), Robert Haas wrote: On Thu, Feb 22, 2018 at 6:54 AM, Etsuro Fujita wrote: Here is a tiny patch to fix $SUBJECT in a comment in execPartition.c. Committed. Thanks! Best regards, Etsuro Fujita

Re: [bug fix] Cascaded standby cannot start after a clean shutdown

2018-02-22 Thread Michael Paquier
On Thu, Feb 22, 2018 at 04:55:38PM +0900, Michael Paquier wrote: > I am definitely ready to buy that it can be possible to have garbage > being read the length field which can cause allocate_recordbuf to fail > as that's the only code path in xlogreader.c which does such an > allocation. Still, it

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Amit Kapila
On Thu, Feb 22, 2018 at 10:35 PM, Robert Haas wrote: > On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro >> PS I noticed that for BecomeLockGroupMember() we say "If we can't >> join the lock group, the leader has gone away, so just exit quietly" >> but for various other similar things we spew errors

Re: FOR EACH ROW triggers on partitioned tables

2018-02-22 Thread Alvaro Herrera
Amit Langote wrote: > On 2018/02/23 8:52, Alvaro Herrera wrote: > > We could mitigate the performance loss to some extent by adding more to > > RelationData. For example, a "is_partition" boolean would help: skip > > searching pg_inherits for a relation that is not a partition. > > Unless I'm miss

Re: Translations contributions urgently needed

2018-02-22 Thread Michael Paquier
On Fri, Feb 23, 2018 at 09:51:02AM +0900, Tatsuo Ishii wrote: > I'm not working on message translations. Hotta-san is actively > working on the translations here: > > https://github.com/hotta/pg-nls-ja If you need help here, feel free to ping me. I have done translation to Japanese for other th

Re: Translations contributions urgently needed

2018-02-22 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Feb 23, 2018 at 09:51:02AM +0900, Tatsuo Ishii wrote: > > I'm not working on message translations. Hotta-san is actively > > working on the translations here: > > > > https://github.com/hotta/pg-nls-ja > > If you need help here, feel free to ping me. I have done

  1   2   >