On Tue, May 25, 2021 at 2:57 PM Dilip Kumar <dilipbal...@gmail.com> wrote:
> > > > > Yes, I am able to reproduce this, basically, until we get the last > > tuple of the multi insert we can not clear the toast data otherwise we > > can never form a complete tuple. So the only possible fix I can think > > of is to consider the multi-insert WAL without the final multi-insert > > tuple as partial data then we will avoid streaming until we get the > > complete WAL of one multi-insert. I am working on the patch to fix > > this, I will share that in some time. > > The attached patch should fix the issue, now the output is like below > > Thanks. This looks fine to me. We should still be able to stream multi-insert transactions (COPY) as and when the copy buffer becomes full and is flushed. That seems to be a reasonable restriction to me. We should incorporate the regression test in the final patch. I am not entirely sure if what I have done is acceptable (or even works in all scenarios). We could possibly have a long list of tuples instead of doing the exponential magic. Or we should consider lowering the min value for logical_decoding_work_mem and run these tests with a much lower value. In fact, that's how I caught the problem in the first place. I had deliberately lowered the value to 1kB so that streaming code kicks in very often and even for small transactions. Thanks, Pavan -- Pavan Deolasee EnterpriseDB: https://www.enterprisedb..com
