Re: [HACKERS] path toward faster partition pruning

2017-11-13 Thread Amit Langote
mment. I'd almost started looking around if something might be wrong after all. :) On 2017/11/10 16:07, Kyotaro HORIGUCHI wrote: > At Fri, 10 Nov 2017 14:44:55 +0900, Kyotaro HORIGUCHI wrote: > >>> Those two conditions are not orthogonal. Maybe something like >>> follow

Re: [HACKERS] Parallel Append implementation

2017-11-12 Thread Amit Khandekar
Thanks a lot Robert for the patch. I will have a look. Quickly tried to test some aggregate queries with a partitioned pgbench_accounts table, and it is crashing. Will get back with the fix, and any other review comments. Thanks -Amit Khandekar On 9 November 2017 at 23:44, Robert Haas wrote

Re: [HACKERS] A GUC to prevent leader processes from running subplans?

2017-11-11 Thread Amit Kapila
_locally? If yes, then I think whatever you said is right. > I've added this to the January Commitfest. > +1 to this idea. Do you think such an option at table level can be meaningful? We have a parallel_workers as a storage option for tables, so users might want leader to pa

Re: [HACKERS] GatherMerge misses to push target list

2017-11-11 Thread Amit Kapila
as what you put into it. > Agreed. Your change looks good to me. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] parallelize queries containing initplans

2017-11-11 Thread Amit Kapila
On Sat, Nov 11, 2017 at 12:15 AM, Robert Haas wrote: > On Tue, Nov 7, 2017 at 4:45 AM, Amit Kapila wrote: >> As mentioned, changed the status of the patch in CF app. > > I spent some time reviewing this patch today and found myself still > quite uncomfortable with the fact

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-10 Thread Amit Kapila
On Fri, Nov 10, 2017 at 9:48 AM, Amit Kapila wrote: > On Fri, Nov 10, 2017 at 8:36 AM, Robert Haas wrote: >> On Thu, Nov 9, 2017 at 9:31 PM, Amit Kapila wrote: >>> Have you set force_parallel_mode=regress; before running the >>> statement? >> >> Yes, I t

Re: [HACKERS] UPDATE of partition key

2017-11-09 Thread Amit Khandekar
On 9 November 2017 at 09:27, Thomas Munro wrote: > On Wed, Nov 8, 2017 at 5:57 PM, Amit Khandekar wrote: >> On 8 November 2017 at 07:55, Thomas Munro >> wrote: >>> On Tue, Nov 7, 2017 at 8:03 AM, Robert Haas wrote: >>>> The changes to trigger.c stil

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-09 Thread Amit Kapila
On Fri, Nov 10, 2017 at 8:36 AM, Robert Haas wrote: > On Thu, Nov 9, 2017 at 9:31 PM, Amit Kapila wrote: >> Have you set force_parallel_mode=regress; before running the >> statement? > > Yes, I tried that first. > >> If so, then why you need to tune other parall

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-09 Thread Amit Kapila
On Fri, Nov 10, 2017 at 12:05 AM, Robert Haas wrote: > On Thu, Nov 9, 2017 at 12:08 AM, Amit Kapila wrote: >> This change looks suspicious to me. I think here we can't use the >> tupDesc constructed from targetlist. One problem, I could see is that >> the c

Re: [HACKERS] Runtime Partition Pruning

2017-11-09 Thread Amit Kapila
ob of this patch to do anything about > that problem. > +1. I think if we really want to do something about plan choice when partitions are involved that should be done as a separate patch. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsq

Re: [HACKERS] path toward faster partition pruning

2017-11-09 Thread Amit Langote
Hi Amul. On 2017/11/09 20:05, amul sul wrote: > On Mon, Nov 6, 2017 at 3:31 PM, Amit Langote > wrote: >> On 2017/11/06 14:32, David Rowley wrote: >>> On 6 November 2017 at 17:30, Amit Langote wrote: >>>> On 2017/11/03 13:32, David Rowley wrote: >>>&g

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-09 Thread Amit Kapila
On Wed, Nov 8, 2017 at 6:48 PM, Robert Haas wrote: > On Wed, Nov 8, 2017 at 7:26 AM, Amit Kapila wrote: >> We do want to generate it later when there isn't inheritance involved, >> but only if there is a single rel involved (simple_rel_array_size >> <=2). The ru

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-08 Thread Amit Kapila
On Wed, Nov 8, 2017 at 1:02 AM, Andres Freund wrote: > Hi, > > On 2017-11-06 10:56:43 +0530, Amit Kapila wrote: >> On Sun, Nov 5, 2017 at 6:54 AM, Andres Freund wrote >> > On 2017-11-05 01:05:59 +0100, Robert Haas wrote: >> >> skip-gather-project-v1.patch do

Re: [HACKERS] [PATCH] fix wrong create table statement in documentation

2017-11-08 Thread Amit Langote
gt; TABLESPACE fasttablespace >> WITH (parallel_workers = 4); > > Yup, that's wrong. Fix pushed, thanks! Oops! Thanks Tom for the fix. Regards, Amit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-08 Thread Amit Kapila
On Wed, Nov 8, 2017 at 4:34 PM, Robert Haas wrote: > On Tue, Nov 7, 2017 at 9:41 PM, Amit Kapila wrote: >> This is required to prohibit generating gather path for top rel in >> case of inheritence (Append node) at this place (we want to generate >> it later when scan/joi

Re: [HACKERS] path toward faster partition pruning

2017-11-08 Thread Amit Langote
Hi Rajkumar, Thanks for testing. On 2017/11/08 15:52, Rajkumar Raghuwanshi wrote: > On Mon, Nov 6, 2017 at 3:31 PM, Amit Langote > wrote: > >> Attached updated set of patches, including the fix to make the new pruning >> code handle Boolean partitioning. >> >

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-07 Thread Amit Langote
if you re-shaped that if/else if test so that it only > performs the loop over the partindexes if it's been set. > > I ended up with the attached version of the function after moving > things around a little bit. Thanks a lot for that. Looks much better now. > I'm sti

Re: [HACKERS] UPDATE of partition key

2017-11-07 Thread Amit Khandekar
patch and check if the test passes ? The patch is to be applied on the main v22 patch. If the test passes, I will include these changes (also for list_parted) in the upcoming v23 patch. Thanks -Amit Khandekar regress_locale_changes.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-07 Thread Amit Kapila
On Wed, Nov 8, 2017 at 2:51 AM, Robert Haas wrote: > On Mon, Nov 6, 2017 at 9:57 PM, Amit Kapila wrote: > >>> Also, even if inheritance is used, we might still be the >>> topmost scan/join target. >> >> Sure, but in that case, it won't generate the gathe

Re: [HACKERS] parallelize queries containing initplans

2017-11-07 Thread Amit Kapila
On Mon, Oct 30, 2017 at 9:00 AM, Amit Kapila wrote: > On Wed, Oct 11, 2017 at 9:24 PM, Robert Haas wrote: >> On Mon, Oct 9, 2017 at 5:56 AM, Amit Kapila wrote: >>> How about always returning false for PARAM_EXTERN? >> >> Yeah, I think that's what we s

Re: [HACKERS] UPDATE of partition key

2017-11-06 Thread Amit Langote
On 2017/11/07 14:40, Amit Khandekar wrote: > On 7 November 2017 at 00:33, Robert Haas wrote: > >> Also, +1 for Amit Langote's idea of trying to merge >> mt_perleaf_childparent_maps with mt_persubplan_childparent_maps. > > Currently I am trying to see if it simpli

Re: [HACKERS] UPDATE of partition key

2017-11-06 Thread Amit Khandekar
On 7 November 2017 at 00:33, Robert Haas wrote: > Also, +1 for Amit Langote's idea of trying to merge > mt_perleaf_childparent_maps with mt_persubplan_childparent_maps. Currently I am trying to see if it simplifies things if we do that. We will be merging these arrays into one,

Re: [HACKERS] Parallel Plans and Cost of non-filter functions

2017-11-06 Thread Amit Kapila
can productively help? You have already helped a lot by providing the use case, but feel free to ping on that thread if you find it is not moving. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-06 Thread Amit Kapila
On Mon, Nov 6, 2017 at 7:05 PM, Robert Haas wrote: > On Mon, Nov 6, 2017 at 11:20 AM, Amit Kapila wrote: >> On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: >>> This looks like it's on the right track to me. I hope Tom will look >>> into it, but if he doesn

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-06 Thread Amit Langote
On 2017/11/06 21:52, David Rowley wrote: > On 6 November 2017 at 23:01, Amit Langote > wrote: >> OK, I have gotten rid of the min/max partition index interface and instead >> adopted the bms_add_range() approach by including your patch to add the >> same in the patch s

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-06 Thread Amit Kapila
On Mon, Nov 6, 2017 at 3:51 AM, Robert Haas wrote: > On Sun, Nov 5, 2017 at 12:57 AM, Amit Kapila wrote: >> Thanks for the confirmation. Find rebased patch attached. > > This looks like it's on the right track to me. I hope Tom will look > into it, but if he does

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-06 Thread Amit Langote
On 2017/11/06 13:15, David Rowley wrote: > On 31 October 2017 at 21:43, Amit Langote > wrote: >> Attached updated version of the patches > > match_clauses_to_partkey() needs to allow for the way quals on Bool > columns are represented. > > create table pt (a bool not

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-05 Thread Amit Kapila
st=0.00..635802.67 > rows=3375405 width=127) (actual time=0.025..649.887 rows=3373722 loops=8) > │ Filter: (l_suppkey > 5012) > │ Rows Removed by Filter: 376252 > │ Planning time: 0.076 ms > │ Execution time: 5986.171 ms > └──── >

Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2017-11-05 Thread Amit Langote
unction too. > > I've read a bit more of the patch and I think even more now that the > min/max stuff should be removed. Oops, I didn't catch this email before sending my earlier reply. Thanks for the bms range patch. Will reply to this shortly after reading your patch and thinking on it a bit. Thanks, Amit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-05 Thread Amit Kapila
On Sat, Nov 4, 2017 at 2:03 AM, Tom Lane wrote: > Michael Paquier writes: >> On Fri, Nov 3, 2017 at 1:10 AM, Amit Kapila wrote: >>> On Fri, Nov 3, 2017 at 2:54 AM, Tom Lane wrote: >>>> I've marked the CF entry closed. However, I'm not sure if we'

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-11-05 Thread Amit Langote
r partitions in the bound order is internal to the server yet, we'd need some new server-side functionality to expose the same with sane SQL-callable interface, which clearly needs its own separate discussion. Thanks, Amit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-05 Thread Amit Langote
On 2017/11/03 6:24, Tom Lane wrote: > Amit Langote writes: >> On 2017/09/26 16:30, Michael Paquier wrote: >>> Cool, let's switch it back to a ready for committer status then. > >> Sure, thanks. > > Pushed with some cosmetic adjustments --- mostly, making

Re: [HACKERS] Parallel Plans and Cost of non-filter functions

2017-11-04 Thread Amit Kapila
tioned by Tom. [1] - https://www.postgresql.org/message-id/CAA4eK1%2B1H5Urm0_Wp-n5XszdLX1YXBqS_zW0f-vvWKwdh3eCJA%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] why not parallel seq scan for slow functions

2017-11-04 Thread Amit Kapila
On Thu, Sep 21, 2017 at 2:35 AM, Jeff Janes wrote: > On Tue, Sep 19, 2017 at 9:15 PM, Amit Kapila > wrote: >> >> On Wed, Sep 20, 2017 at 3:05 AM, Jeff Janes wrote: >> > On Tue, Sep 19, 2017 at 1:17 PM, Thomas Munro >> > wrote: >> >> &g

Re: [HACKERS] parallelize queries containing initplans

2017-11-03 Thread Amit Kapila
On Mon, Oct 30, 2017 at 10:07 AM, Robert Haas wrote: > On Mon, Oct 30, 2017 at 9:00 AM, Amit Kapila wrote: >> Now that the PARAM_EXTERN issue is fixed, I have rebased this patch. >> This patch had been switched to Ready For Committer in last CF, then >> Robert had comments w

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-11-02 Thread Amit Kapila
On Fri, Nov 3, 2017 at 2:54 AM, Tom Lane wrote: > Amit Langote writes: >> On 2017/09/26 16:30, Michael Paquier wrote: >>> Cool, let's switch it back to a ready for committer status then. > >> Sure, thanks. > > Pushed with some cosmetic adjustments --- mostl

Re: [HACKERS] UPDATE of partition key

2017-11-02 Thread Amit Langote
Hi Amit. Thanks a lot for updated patches and sorry that I couldn't get to looking at your emails sooner. Note that I'm replying here to both of your emails, but looking at only the latest v22 patch. On 2017/10/24 0:15, Amit Khandekar wrote: > On 16 October 2017 at 08:28, Amit Lang

Re: [HACKERS] Adding column_constraint description in ALTER TABLE synopsis

2017-10-31 Thread Amit Langote
On 2017/10/31 21:31, Stephen Frost wrote: > * Lætitia Avrot (laetitia.av...@gmail.com) wrote: >> As Amit Langot pointed out, the column_constraint definition is missing >> whereas it is used in ALTER TABLE synopsis. It can be easily found in the >> CREATE TABLE synopsis, b

Re: [HACKERS] path toward faster partition pruning

2017-10-31 Thread Amit Langote
und bugs in 0003 and 0005 that caused this. Will post the patches containing the fix in reply to the Dilip's email which contains some code review comments [1]. Also, I noticed that the new pruning code was having a hard time do deal with the fact that the default "range&

Re: [HACKERS] Parallel worker error

2017-10-30 Thread Amit Kapila
On Mon, Oct 30, 2017 at 10:04 AM, Robert Haas wrote: > On Mon, Oct 30, 2017 at 8:25 AM, Amit Kapila wrote: >> Thanks. I have closed this entry in CF app, however, I am not sure >> what is the best way to deal with the entry present in PostgreSQL 10 >> Open Items page[1]. T

Re: [HACKERS] parallelize queries containing initplans

2017-10-29 Thread Amit Kapila
On Wed, Oct 18, 2017 at 2:06 PM, tushar wrote: > On 10/11/2017 12:42 PM, tushar wrote: >> >> On 10/09/2017 03:26 PM, Amit Kapila wrote: >>> >>> I have reverted the check >>> in the attached patch. >> >> >> I have applied this patch again

Re: [HACKERS] parallelize queries containing initplans

2017-10-29 Thread Amit Kapila
On Wed, Oct 11, 2017 at 9:24 PM, Robert Haas wrote: > On Mon, Oct 9, 2017 at 5:56 AM, Amit Kapila wrote: >> How about always returning false for PARAM_EXTERN? > > Yeah, I think that's what we should do. Let's do that first as a > separate patch, which we might e

Re: [HACKERS] Parallel worker error

2017-10-29 Thread Amit Kapila
On Sun, Oct 29, 2017 at 1:15 PM, Robert Haas wrote: > On Sun, Oct 29, 2017 at 12:02 PM, Amit Kapila wrote: >> This patch no longer applies, so attached rebased patches. I have >> also created patches for v10 as we might want to backpatch the fix. >> Added t

Re: [HACKERS] Parallel safety for extern params

2017-10-29 Thread Amit Kapila
On Sun, Oct 29, 2017 at 9:55 PM, Robert Haas wrote: > On Sat, Oct 28, 2017 at 8:02 PM, Amit Kapila wrote: >> I think we need to make changes in exec_simple_recheck_plan to make >> the behavior similar to HEAD. With the attached patch, all tests >> passed with force_paralle

Re: [HACKERS] Parallel worker error

2017-10-28 Thread Amit Kapila
On Sat, Sep 9, 2017 at 7:56 AM, Amit Kapila wrote: > On Fri, Sep 8, 2017 at 3:13 PM, Robert Haas wrote: >> On Fri, Sep 8, 2017 at 1:17 AM, Amit Kapila wrote: >>> You are right. I have changed the ordering and passed OuterUserId via >>> FixedParallelState. >&g

Re: [HACKERS] Parallel safety for extern params

2017-10-28 Thread Amit Kapila
On Sat, Oct 28, 2017 at 2:02 AM, Robert Haas wrote: > On Mon, Oct 16, 2017 at 12:59 PM, Amit Kapila wrote: > When I tried back-porting the patch to v10 I discovered that the > plpgsql changes conflict heavily and that ripping them out all the way > produces regression fa

Re: [HACKERS] Re: Burst in WAL size when UUID is used as PK while full_page_writes are enabled

2017-10-27 Thread Amit Kapila
because hash indexes > don't support uniqueness. > That's true, but it hasn't been mentioned in the mail that the usage of hash index is the for primary key. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] Burst in WAL size when UUID is used as PK while full_page_writes are enabled

2017-10-27 Thread Amit Kapila
On Fri, Oct 27, 2017 at 5:36 PM, Alvaro Herrera wrote: > Amit Kapila wrote: > >> You might want to give a try with the hash index if you are planning >> to use PG10 and your queries involve equality operations. > > So, btree indexes on monotonically increasing sequences do

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2017-10-27 Thread Amit Kapila
On Tue, Sep 19, 2017 at 8:47 AM, Amit Kapila wrote: > On Mon, Sep 18, 2017 at 10:00 PM, Tom Lane wrote: >> Amit Kapila writes: >>> Attached patch fixes these problems. >> >> Hmm, this patch adds a kill(notify_pid) after one call to >> ForgetBackgroundWorker,

Re: [HACKERS] Burst in WAL size when UUID is used as PK while full_page_writes are enabled

2017-10-27 Thread Amit Kapila
want to give a try with the hash index if you are planning to use PG10 and your queries involve equality operations. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] Parallel safety for extern params

2017-10-27 Thread Amit Kapila
On Mon, Oct 16, 2017 at 4:29 PM, Amit Kapila wrote: > On Sat, Oct 14, 2017 at 1:51 AM, Robert Haas wrote: > >> I think the bug is in ExecGather(Merge): it assumes that if we're in >> parallel mode, it's OK to start workers. But actually, it shouldn't >> d

Re: [HACKERS] path toward faster partition pruning

2017-10-26 Thread Amit Langote
On 2017/10/27 13:57, Robert Haas wrote: > On Fri, Oct 27, 2017 at 3:17 AM, Amit Langote > wrote: >>> I don't think we really want to get into theorem-proving here, because >>> it's slow. >> >> Just to be clear, I'm saying we could use theorem-pr

Re: [HACKERS] path toward faster partition pruning

2017-10-26 Thread Amit Langote
On 2017/10/26 20:34, Robert Haas wrote: > On Thu, Oct 26, 2017 at 1:17 PM, Amit Langote > wrote: >> It can perhaps taught to not make that conclusion by taking into account >> the default partition's partition constraint, which includes constraint >> inherited fr

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

2017-10-26 Thread Amit Kapila
nce that this is some of your application issues where you probably haven't used memory context as required, so probably you need to figure out a way to reproduce this issue, otherwise, it might be difficult to track down the actual cause. -- With Regards, Amit Kapila. EnterpriseDB: http

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

2017-10-25 Thread Amit Kapila
Context shouldn't get freed > before transaction COMMIT. >--> So what has actually happened here??? Kindly help us with > your insights! > > Can you check if CurTransactionContext is valid at that point? To see, if this problem is related to CurTransactionContext, can you try to populate the list in TopMemoryContext and see if that works. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Pluggable storage

2017-10-25 Thread Amit Kapila
On Wed, Oct 25, 2017 at 11:37 AM, Robert Haas wrote: > On Fri, Oct 20, 2017 at 5:47 AM, Amit Kapila wrote: >> I think what we need here is a way to register satisfies function >> (SnapshotSatisfiesFunc) in SnapshotData for different storage engines. > > I don't se

Re: [HACKERS] BLK_DONE state in XLogReadBufferForRedoExtended

2017-10-23 Thread Amit Kapila
On Tue, Oct 17, 2017 at 7:53 AM, Michael Paquier wrote: > On Mon, Oct 16, 2017 at 9:50 PM, Amit Kapila wrote: >> If above analysis is correct, then I think we can say that row state >> in a page will be same during recovery as it was when the original >> operation

Re: [HACKERS] [sqlsmith] Failed assertion in adjust_appendrel_attrs_mutator

2017-10-23 Thread Amit Langote
On 2017/10/24 0:22, Tom Lane wrote: > Amit Langote writes: >> On 2017/10/23 2:07, Tom Lane wrote: >>> Hmm. adjust_appendrel_attrs() thinks it's only used after conversion >>> of sublinks to subplans, but this is a counterexample. I wonder if >>> that

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-10-23 Thread Amit Langote
ascading index feature, at least, >> committed to v11. That's already a considerable step forward in terms >> of ease of use, and I'd really like to have it. > > Absolutely -- I do plan to get this one finished regardless of unique > indexes. +1 Thanks, Amit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] Faster processing at Gather node

2017-10-23 Thread Amit Kapila
milar where the tuples are sent till there is a space in shared memory queue and then turn to batching the tuples using local queues. [1] - https://www.postgresql.org/message-id/CAOGQiiNiMhq5Pg3LiYxjfi2B9eAQ_q5YjS%3DfHiBJmbSOF74aBQ%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB:

Re: [HACKERS] [POC] Faster processing at Gather node

2017-10-23 Thread Amit Kapila
the structural issues, instead opting to > address them by adding another layer of queuing. > That is done to use batching the tuples while sending them. Sure, we can do some of the other things as well, but I think the main advantage is from batching the tuples in a smart way while sending them

Re: [HACKERS] [sqlsmith] Failed assertion in adjust_appendrel_attrs_mutator

2017-10-23 Thread Amit Langote
omething. The attached patch to do the same, while didn't break any existing tests, fixed the crash reported by OP. Thoughts? Thanks, Amit diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c index 1c84a2cb28..4bdfe73d29 100644 --- a/src/backend/op

Re: [HACKERS] alter table doc fix

2017-10-22 Thread Amit Langote
On 2017/10/18 20:37, Alvaro Herrera wrote: > Amit Langote wrote: >> Hi. >> >> Noticed that a alter table sub-command's name in Description (where it's >> OWNER) differs from that in synopsis (where it's OWNER TO). Attached >> patch to make them m

Re: [HACKERS] Block level parallel vacuum WIP

2017-10-22 Thread Amit Langote
s to prevent the "cache lookup failed for relation XXX" error in a different code path though (the code path handling manual vacuum). Not sure if the commit could have prevented that error being emitted by DROP SCHEMA ... CASCADE, which seems to be what produced it in this

Re: [HACKERS] Pluggable storage

2017-10-19 Thread Amit Kapila
lanQual mechanism and then later used during the scan. We need to make it pluggable in some way so that other heaps can work. We also need some work for EvalPlanQualFetchRowMarks as that also seems to be tightly coupled with HeapTuple. -- With Regards, Amit Kapila. EnterpriseDB: http://www.en

Re: [HACKERS] Pluggable storage

2017-10-19 Thread Amit Kapila
eap it will do >> heap_hot_search_buffer, and for zheap it will walk the undo chain and >> return the relevant tuple from the chain. > > > OK, Understood. > I will check along these lines and come up with storage API's. > I think what we need here is a

Re: [HACKERS] Parallel Append implementation

2017-10-19 Thread Amit Khandekar
On 13 October 2017 at 00:29, Robert Haas wrote: > On Wed, Oct 11, 2017 at 8:51 AM, Amit Khandekar > wrote: >> [ new patch ] > > + parallel_append > + Waiting to choose the next subplan during Parallel Append > plan > + execution. > +

Re: [HACKERS] [COMMITTERS] pgsql: Implement table partitioning.

2017-10-17 Thread Amit Langote
of recursive application of ownership change to descendant tables is unintentional. > The alter table docs say that ONLY must be specified if one does not > want to modify descendants, so I think this is a bug. Just to clarify, if we do think of it as a bug, then it will apply to the inheritance cas

[HACKERS] alter table doc fix

2017-10-17 Thread Amit Langote
Hi. Noticed that a alter table sub-command's name in Description (where it's OWNER) differs from that in synopsis (where it's OWNER TO). Attached patch to make them match, if the difference is unintentional. Thanks, Amit diff --git a/doc/src/sgml/ref/alter_table.sgml b/d

Re: [HACKERS] BLK_DONE state in XLogReadBufferForRedoExtended

2017-10-16 Thread Amit Kapila
On Fri, Oct 13, 2017 at 11:57 AM, Amit Kapila wrote: > On Fri, Oct 13, 2017 at 10:32 AM, Michael Paquier > wrote: >> On Thu, Oct 12, 2017 at 10:47 PM, Amit Kapila >> wrote: >>> Today, I was trying to think about cases when we can return BLK_DONE >>> in XLogR

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-16 Thread Amit Kapila
> One idea could be that whenever someone uses Parallel option, we can fetch and store all the data locally before returning control to the user (something like WITH HOLD option). -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] Parallel safety for extern params

2017-10-16 Thread Amit Kapila
On Sat, Oct 14, 2017 at 1:51 AM, Robert Haas wrote: > On Fri, Oct 13, 2017 at 1:19 AM, Amit Kapila wrote: >> After fixing this problem, when I ran the regression tests with >> force_parallel_mode = regress, I saw multiple other failures. All the >> failures boil down

Re: [HACKERS] relkind check in DefineIndex

2017-10-15 Thread Amit Langote
On 2017/10/14 4:32, Robert Haas wrote: > On Fri, Oct 13, 2017 at 12:38 PM, Alvaro Herrera > wrote: >> The relkind check in DefineIndex has grown into an ugly rats nest of >> 'if' statements. I propose to change it into a switch, as per the >> attached. > &g

Re: [HACKERS] UPDATE of partition key

2017-10-15 Thread Amit Langote
Hi Amit. On 2017/10/04 22:51, Amit Khandekar wrote: > Main patch : > update-partition-key_v20.patch Guess you're already working on it but the patch needs a rebase. A couple of hunks in the patch to execMain.c and nodeModifyTable.c fail. Meanwhile a few comme

Re: [HACKERS] v10 bottom-listed

2017-10-15 Thread Amit Langote
On 2017/10/13 22:58, Magnus Hagander wrote: > On Fri, Oct 6, 2017 at 3:59 AM, Amit Langote > wrote: > >> On 2017/10/05 22:28, Erik Rijkers wrote: >>> In the 'ftp' listing, v10 appears at the bottom: >>> https://www.postgresql.org/ftp/source/ >>

Re: [HACKERS] pgsql: Avoid coercing a whole-row variable that is already coerced

2017-10-13 Thread Amit Khandekar
hack. I'm not sure whether this case ever arises currently, but the pending patch for update tuple routing will cause it to arise. Amit Khandekar Discussion: http://postgr.es/m/caj3gd9cazfppe7-wwubabpcq4_0subkipfd1+0r5_dkvnwo...@mail.gmail.com Tom Lane wrote: > Robert Haa

Re: [HACKERS] BLK_DONE state in XLogReadBufferForRedoExtended

2017-10-12 Thread Amit Kapila
On Fri, Oct 13, 2017 at 10:32 AM, Michael Paquier wrote: > On Thu, Oct 12, 2017 at 10:47 PM, Amit Kapila wrote: >> Today, I was trying to think about cases when we can return BLK_DONE >> in XLogReadBufferForRedoExtended. One thing that occurred to me is >> that it can hap

Re: [HACKERS] parallelize queries containing initplans

2017-10-12 Thread Amit Kapila
On Wed, Oct 11, 2017 at 9:24 PM, Robert Haas wrote: > On Mon, Oct 9, 2017 at 5:56 AM, Amit Kapila wrote: >> How about always returning false for PARAM_EXTERN? > > Yeah, I think that's what we should do. Let's do that first as a > separate patch, which we might e

[HACKERS] Parallel safety for extern params

2017-10-12 Thread Amit Kapila
on_v1.patch fixes this problem. Thanks to Kuntal who has helped in investigating the regression failures which happened as a result of making param extern params as parallel-safe. [1] - https://www.postgresql.org/message-id/CA%2BTgmobSN66o2_c76rY12JvS_sZa17zpKvpuyG-QzRvVLYE8Vg%40mail.gmail.com -

Re: [HACKERS] UPDATE of partition key

2017-10-12 Thread Amit Langote
On 2017/10/13 6:18, Robert Haas wrote: > Is anybody still reviewing the main patch here? (It would be good if > the answer is "yes".) I am going to try to look at the latest version over the weekend and early next week. Thanks, Amit -- Sent via pgsql-hackers mailing lis

Re: [HACKERS] Optimise default partition scanning while adding new partition

2017-10-12 Thread Amit Langote
On 2017/10/13 4:18, Robert Haas wrote: > On Thu, Oct 5, 2017 at 9:29 PM, Amit Langote > wrote: >> Attached a patch to modify the INFO messages in check_default_allows_bound. > > Committed. However, I didn't see a reason to adopt the comment change > you proposed, so I

[HACKERS] BLK_DONE state in XLogReadBufferForRedoExtended

2017-10-12 Thread Amit Kapila
am missing? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-10-11 Thread Amit Langote
On 2017/09/30 1:53, Robert Haas wrote: > On Thu, Sep 28, 2017 at 1:54 AM, Amit Langote > wrote: >> I looked into how satisfies_hash_partition() works and came up with an >> idea that I think will make constraint exclusion work. What if we emitted >> the hash partition con

Re: [HACKERS] Parallel Append implementation

2017-10-11 Thread Amit Khandekar
On 9 October 2017 at 16:03, Amit Kapila wrote: > On Fri, Oct 6, 2017 at 12:03 PM, Amit Khandekar > wrote: >> On 6 October 2017 at 08:49, Amit Kapila wrote: >>> >>> Okay, but why not cheapest partial path? >> >> I gave some thought on this point.

Re: [HACKERS] Parallel Append implementation

2017-10-09 Thread Amit Kapila
On Fri, Oct 6, 2017 at 12:03 PM, Amit Khandekar wrote: > On 6 October 2017 at 08:49, Amit Kapila wrote: >> >> Okay, but why not cheapest partial path? > > I gave some thought on this point. Overall I feel it does not matter > which partial path it should pick up. RIght n

Re: [HACKERS] parallelize queries containing initplans

2017-10-09 Thread Amit Kapila
On Sat, Oct 7, 2017 at 7:22 PM, Robert Haas wrote: > On Fri, Oct 6, 2017 at 7:08 AM, Amit Kapila wrote: >> I have fixed the other review comment related to using safe_param_list >> in the attached patch. I think I have fixed all comments given by >> you, but let me

Re: [HACKERS] parallelize queries containing initplans

2017-10-06 Thread Amit Kapila
On Fri, Oct 6, 2017 at 2:32 AM, Robert Haas wrote: > On Thu, Oct 5, 2017 at 1:16 PM, Amit Kapila wrote: >> On Thu, Oct 5, 2017 at 6:08 PM, Robert Haas wrote: >>> On Thu, Oct 5, 2017 at 5:52 AM, Amit Kapila wrote: >>>> Now, unless, I am missing something

Re: [HACKERS] Parallel Append implementation

2017-10-05 Thread Amit Khandekar
On 6 October 2017 at 08:49, Amit Kapila wrote: > On Thu, Oct 5, 2017 at 4:11 PM, Amit Khandekar wrote: >> >> Ok. How about removing pa_all_partial_subpaths altogether , and >> instead of the below condition : >> >> /* >> * If all the child rels have p

Re: [HACKERS] Parallel Append implementation

2017-10-05 Thread Amit Kapila
On Thu, Oct 5, 2017 at 4:11 PM, Amit Khandekar wrote: > > Ok. How about removing pa_all_partial_subpaths altogether , and > instead of the below condition : > > /* > * If all the child rels have partial paths, and if the above Parallel > * Append path has a mix of partial and

Re: [HACKERS] v10 bottom-listed

2017-10-05 Thread Amit Langote
be fixed so that it appears at the top. Still see it at the bottom. Maybe ping pgsql-www? Thanks, Amit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Optimise default partition scanning while adding new partition

2017-10-05 Thread Amit Langote
On 2017/10/06 2:25, Robert Haas wrote: > On Tue, Sep 26, 2017 at 4:27 AM, Amit Langote wrote: >> I guess we don't need to squash, as they could be seen as implementing >> different features. Reordering the patches helps though. So, apply them >> in this order:

Re: [HACKERS] parallelize queries containing initplans

2017-10-05 Thread Amit Kapila
On Thu, Oct 5, 2017 at 6:08 PM, Robert Haas wrote: > On Thu, Oct 5, 2017 at 5:52 AM, Amit Kapila wrote: >> Now, unless, I am missing something here, it won't be possible to >> detect params in such cases during forming of join rels and hence we >> need the tests in gene

Re: [HACKERS] Parallel Append implementation

2017-10-05 Thread Amit Kapila
On Thu, Oct 5, 2017 at 6:14 PM, Robert Haas wrote: > On Thu, Oct 5, 2017 at 6:29 AM, Amit Kapila wrote: >> Okay, but can't we try to pick the cheapest partial path for master >> backend and maybe master backend can try to work on a partial path >> which is alread

Re: [HACKERS] Parallel Append implementation

2017-10-05 Thread Amit Khandekar
On 30 September 2017 at 19:21, Amit Kapila wrote: > On Wed, Sep 20, 2017 at 10:59 AM, Amit Khandekar > wrote: >> On 16 September 2017 at 10:42, Amit Kapila wrote: >>> >>> At a broader level, the idea is good, but I think it won't turn out >>> exactl

Re: [HACKERS] Parallel Append implementation

2017-10-05 Thread Amit Kapila
On Mon, Oct 2, 2017 at 6:21 PM, Robert Haas wrote: > On Sun, Oct 1, 2017 at 9:55 AM, Amit Kapila wrote: >> Isn't it for both? I mean it is about comparing the non-partial paths >> for child relations of the same relation and also when there are >> different relation

Re: [HACKERS] parallelize queries containing initplans

2017-10-05 Thread Amit Kapila
On Wed, Oct 4, 2017 at 12:55 PM, Amit Kapila wrote: > On Wed, Oct 4, 2017 at 3:40 AM, Robert Haas wrote: >> On Tue, Oct 3, 2017 at 7:33 AM, Amit Kapila wrote: >> >> Having said all that, I think that this patch only wants to handle the >> subset of cases (2) and (4)

Re: [HACKERS] parallelize queries containing initplans

2017-10-04 Thread Amit Kapila
On Wed, Oct 4, 2017 at 12:55 PM, Amit Kapila wrote: > On Wed, Oct 4, 2017 at 3:40 AM, Robert Haas wrote: >> On Tue, Oct 3, 2017 at 7:33 AM, Amit Kapila wrote: >> >> Having said all that, I think that this patch only wants to handle the >> subset of cases (2) and (4)

Re: [HACKERS] parallelize queries containing initplans

2017-10-04 Thread Amit Kapila
On Wed, Oct 4, 2017 at 3:40 AM, Robert Haas wrote: > On Tue, Oct 3, 2017 at 7:33 AM, Amit Kapila wrote: > > Having said all that, I think that this patch only wants to handle the > subset of cases (2) and (4) where the relevant InitPlan is attached > ABOVE the Gather node -- w

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-03 Thread Amit Langote
FWIW, the name enable_partition_join seems enough to convey the core feature, that is, I see "_wise" as redundant, even though I'm now quite used to seeing "_wise" in the emails here and saying it out loud every now and then. Ashutosh may have a point though that users comi

Re: [HACKERS] UPDATE of partition key

2017-10-03 Thread Amit Khandekar
On 30 September 2017 at 01:26, Robert Haas wrote: > On Fri, Sep 29, 2017 at 3:53 PM, Robert Haas wrote: >> On Fri, Sep 22, 2017 at 1:57 AM, Amit Khandekar >> wrote: >>> The patch for the above change is : >>> 0002-Prevent-a-redundant-ConvertRowtypeExpr-node.

  1   2   3   4   5   6   7   8   9   10   >