On 2018-Aug-06, Kohei KaiGai wrote:
> 2018-08-06 1:50 GMT+09:00 Alvaro Herrera :
> >> Now, it consumed about 60MB rss at the beginning of COPY FROM, and it
> >> grows up very slowly during the COPY FROM execution, then grew up to
> >> 250MB before completion.
> >> We may have another memory blocks
2018-08-06 1:50 GMT+09:00 Alvaro Herrera :
>> Now, it consumed about 60MB rss at the beginning of COPY FROM, and it
>> grows up very slowly during the COPY FROM execution, then grew up to
>> 250MB before completion.
>> We may have another memory blocks which are not released during
>> execution, ho
On 2018-Aug-04, Kohei KaiGai wrote:
> I could load the same data (544GB csv, 789GB heap) using COPY FROM
> successfully.
> When I reported the problem, rss usage of postgresql process increased
> about 10MB/s ratio, then OOM killer eliminated after a few hours.
OK, I think we can consider this p
2018-08-03 12:38 GMT+09:00 Alvaro Herrera :
> On 2018-Aug-03, Kohei KaiGai wrote:
>
>> 2018-08-02 5:38 GMT+09:00 Alvaro Herrera :
>> > On 2018-Aug-01, Alvaro Herrera wrote:
>> >
>> >> Right, makes sense. Pushed that way.
>> >
>> > KaiGai, if you can please confirm that the pushed change fixes your
On 2018-Aug-03, Kohei KaiGai wrote:
> 2018-08-02 5:38 GMT+09:00 Alvaro Herrera :
> > On 2018-Aug-01, Alvaro Herrera wrote:
> >
> >> Right, makes sense. Pushed that way.
> >
> > KaiGai, if you can please confirm that the pushed change fixes your test
> > case, I'd appreciate it.
>
> Can you wait f
2018-08-02 5:38 GMT+09:00 Alvaro Herrera :
> On 2018-Aug-01, Alvaro Herrera wrote:
>
>> Right, makes sense. Pushed that way.
>
> KaiGai, if you can please confirm that the pushed change fixes your test
> case, I'd appreciate it.
>
Can you wait for a few days? I can drop the test dataset and reuse
On 2018-Aug-02, Amit Langote wrote:
> On 2018/08/02 6:03, Alvaro Herrera wrote:
> > Hmm, variable shadowing ...
>
> Crap, sorry about forgetting to remove that and thanks for taking care of
> that.
I think it was my bug actually, while rebasing to branch master after
renaming the variable.
--
On 2018/08/02 6:03, Alvaro Herrera wrote:
> On 2018-Aug-01, Andres Freund wrote:
>
>> On 2018-08-01 16:38:11 -0400, Alvaro Herrera wrote:
>>> On 2018-Jul-24, Amit Langote wrote:
>>>
Your patch takes care of allocation happening inside
get_partition_for_tuple, but as you mention there mig
On 2018/08/02 5:38, Alvaro Herrera wrote:
> On 2018-Jul-24, Amit Langote wrote:
>
>> Your patch takes care of allocation happening inside
>> get_partition_for_tuple, but as you mention there might be others in its
>> caller ExecFindPartition. So, I think we should switch to the per-tuple
>> conte
On 2018-Aug-01, Andres Freund wrote:
> On 2018-08-01 16:38:11 -0400, Alvaro Herrera wrote:
> > On 2018-Jul-24, Amit Langote wrote:
> >
> > > Your patch takes care of allocation happening inside
> > > get_partition_for_tuple, but as you mention there might be others in its
> > > caller ExecFindPar
On 2018-08-01 16:38:11 -0400, Alvaro Herrera wrote:
> On 2018-Jul-24, Amit Langote wrote:
>
> > Your patch takes care of allocation happening inside
> > get_partition_for_tuple, but as you mention there might be others in its
> > caller ExecFindPartition. So, I think we should switch to the per-t
On 2018-Aug-01, Alvaro Herrera wrote:
> Right, makes sense. Pushed that way.
KaiGai, if you can please confirm that the pushed change fixes your test
case, I'd appreciate it.
--
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Trainin
On 2018-Jul-24, Amit Langote wrote:
> Your patch takes care of allocation happening inside
> get_partition_for_tuple, but as you mention there might be others in its
> caller ExecFindPartition. So, I think we should switch to the per-tuple
> context in ExecFindPartition.
Right, makes sense. Pus
Hi Kaigai-san,
Thanks for the report and the patch.
On 2018/07/24 11:43, Kohei KaiGai wrote:
> Further investigation I did:
>
> CopyFrom() calls ExecFindPartition() to identify the destination child
> table of partitioned table.
> Then, it internally calls get_partition_for_tuple() to get partit
Further investigation I did:
CopyFrom() calls ExecFindPartition() to identify the destination child
table of partitioned table.
Then, it internally calls get_partition_for_tuple() to get partition
index according to the key value.
This invocation is not under the per-tuple context.
In case of has
On Tue, Jul 24, 2018 at 09:41:52AM +0900, Kohei KaiGai wrote:
> In PG11beta2, my backend process gets terminated during COPY FROM of
> large text file (544GB) on partitioned table.
> The kernel log says OOM Killer send SIGKILL due to memory pressure.
> In fact, 63GB of system physical 64GB was cons
Hello,
In PG11beta2, my backend process gets terminated during COPY FROM of
large text file (544GB) on partitioned table.
The kernel log says OOM Killer send SIGKILL due to memory pressure.
In fact, 63GB of system physical 64GB was consumed by the PostgreSQL
backend just before the termination.
O
17 matches
Mail list logo
