Re: faster ETL / bulk data load for heap tables

2021-01-03 Thread Amit Kapila
On Sat, Jan 2, 2021 at 3:34 PM Luc Vlaming wrote: > > On 02-01-2021 08:36, Amit Kapila wrote: > > On Fri, Jan 1, 2021 at 7:37 PM Luc Vlaming wrote: > >> > >> Hi, > >> > >> In an effort to speed up bulk data loading/transforming I noticed that > >> considerable time is spent in the relation extens

Re: faster ETL / bulk data load for heap tables

2021-01-02 Thread Luc Vlaming
On 01-01-2021 19:55, Zhihong Yu wrote: Hi, Luc: Happy New Year. Looking at BufferAllocExtend() in v1-0002-WIP-buffer-alloc-specialized-for-relation-extensi.patch. it seems there is duplicate code with the existing BufferAlloc(). It would be good if some refactoring is done by extracting comm

Re: faster ETL / bulk data load for heap tables

2021-01-02 Thread Luc Vlaming
On 02-01-2021 08:36, Amit Kapila wrote: On Fri, Jan 1, 2021 at 7:37 PM Luc Vlaming wrote: Hi, In an effort to speed up bulk data loading/transforming I noticed that considerable time is spent in the relation extension lock. We already do extend the relation in bulk when there is a contenti

Re: faster ETL / bulk data load for heap tables

2021-01-01 Thread Amit Kapila
On Fri, Jan 1, 2021 at 7:37 PM Luc Vlaming wrote: > > Hi, > > In an effort to speed up bulk data loading/transforming I noticed that > considerable time is spent in the relation extension lock. > We already do extend the relation in bulk when there is a contention on relation extension lock via R

Re: faster ETL / bulk data load for heap tables

2021-01-01 Thread Zhihong Yu
Hi, Luc: Happy New Year. Looking at BufferAllocExtend() in v1-0002-WIP-buffer-alloc-specialized-for-relation-extensi.patch. it seems there is duplicate code with the existing BufferAlloc(). It would be good if some refactoring is done by extracting common code into a helper function. Thanks On

faster ETL / bulk data load for heap tables

2021-01-01 Thread Luc Vlaming
Hi, In an effort to speed up bulk data loading/transforming I noticed that considerable time is spent in the relation extension lock. I know there are already many other efforts to increase the chances of using bulk loading [1], [2], [3], [4], efforts to make loading more parallel [5], and re