ll the RI columns are there in the column list (for
update/delete) and we don't need to apply a column filter for old
tuples in either update or delete.
We can remove this FIXME.
11.
+ } /* loop all subscribed publications */
+
+}
No need for an empty line here.
[1] -
https://www.postgresql.org/message-id/CAA4eK1K5pkrPT9z5TByUPptExian5c18g6GnfNf9Cr97QdPbjw%40mail.gmail.com
[2] -
https://www.postgresql.org/message-id/43c15aa8-aa15-ca0f-40e4-3be68d98df05%40enterprisedb.com
--
With Regards,
Amit Kapila.
> > > wrote:
> > > >
> > > > On Sat, Mar 12, 2022 at 3:15 PM Andres Freund
> > > > wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > On 2022-03-12 08:28:35 +0530, Amit Kapila wrote:
> >
and SKIP transaction features."?
I am making some other minor edits in the patch and will take care of
whatever we decide for these comments.
--
With Regards,
Amit Kapila.
eek (on Monday) unless there
are more comments/suggestions.
--
With Regards,
Amit Kapila.
v16-0001-Add-ALTER-SUBSCRIPTION-.-SKIP.patch
Description: Binary data
nge, we would not need to worry about the overhead much.
>
But won't that lead to a call to GetCurrentTimestamp() for each change
we skip? IIUC from previous replies that lead to a slight slowdown in
previous tests of Wang-San.
> Actually, WalSndWriteData() does similar things;
>
That also every time seems to be calling GetCurrentTimestamp(). I
think it might be okay when we are sending the change but not sure if
the overhead of the same is negligible when we are skipping the
changes.
--
With Regards,
Amit Kapila.
On Thu, Mar 17, 2022 at 12:39 PM osumi.takami...@fujitsu.com
wrote:
>
> On Thursday, March 17, 2022 3:04 PM Amit Kapila
> wrote:
> > On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada
> > wrote:
> > >
> > > I've attached an updated version patch.
>
On Thu, Mar 17, 2022 at 12:27 PM Amit Kapila wrote:
>
> On Wed, Mar 16, 2022 at 7:38 PM Masahiko Sawada wrote:
> >
> > After more thought, can we check only wal_sender_timeout without
> > skip-count? That is, in WalSndUpdateProgress(), if we have received
> > an
ion parameter as discussed
[1]) as discussed in this thread pgoutput.c?
3. Can we add a simple test for it in one of the existing test
files(say in 001_rep_changes.pl)?
4. I think we can drop the skip streaming patch as we can't do that for now.
--
With Regards,
Amit Kapila.
e-id/CAA4eK1KR%2ByUQquK0Bx9uO3eb5xB1e0rAD9xKf-ddm5nSf4WfNg%40mail.gmail.com
[3] -
https://www.postgresql.org/message-id/TYAPR01MB6315D664D926EF66DD6E91FCFD109%40TYAPR01MB6315.jpnprd01.prod.outlook.com
--
With Regards,
Amit Kapila.
_sender_timeout / 2) &&
!pq_is_send_pending())
{
return;
}
* Can we rename variable 'is_send' to 'change_sent'?
--
With Regards,
Amit Kapila.
r
that tablesync worker won't be able to complete because in our case
apply worker won't be able to restart.
> So the question is why those two sync workers never complete - I guess
> there's some sort of lock wait (deadlock?) or infinite loop.
>
It would be a bit tricky to reproduce this even if the above theory is
correct but I'll try it today or tomorrow.
--
With Regards,
Amit Kapila.
On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote:
>
> On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra
> wrote:
>
> > So the question is why those two sync workers never complete - I guess
> > there's some sort of lock wait (deadlock?) or infinite loop.
> &
also name 029_on_error.pl to something else such as 030_skip_lsn.pl
> or
> a generic name 030_skip_option.pl.
>
The reason to keep the name 'on_error' is that it has tests for both
'disable_on_error' option and 'skip_lsn'. The other option could be
'on_error_action' or something like that. Now, does this make sense to
you?
--
With Regards,
Amit Kapila.
On Mon, Mar 21, 2022 at 7:09 AM Euler Taveira wrote:
>
> On Thu, Mar 17, 2022, at 3:03 AM, Amit Kapila wrote:
>
> On Wed, Mar 16, 2022 at 1:53 PM Masahiko Sawada wrote:
> >
> > I've attached an updated version patch.
> >
>
> The patch LGTM. I have made
On Fri, Mar 18, 2022 at 4:20 PM Amit Kapila wrote:
>
> On Fri, Mar 18, 2022 at 10:43 AM wangw.f...@fujitsu.com
> wrote:
> >
> > On Thu, Mar 17, 2022 at 7:52 PM Masahiko Sawada
> > wrote:
> > >
> >
> > Attach the new patch.
> >
>
> *
On Sat, Mar 19, 2022 at 9:10 AM Ajin Cherian wrote:
>
> On Thu, Mar 17, 2022 at 10:43 PM Amit Kapila wrote:
>
> > 3. Can we add a simple test for it in one of the existing test
> > files(say in 001_rep_changes.pl)?
>
> added a simple test.
>
This doesn't veri
On Sun, Mar 20, 2022 at 4:53 PM Tomas Vondra
wrote:
>
> On 3/20/22 07:23, Amit Kapila wrote:
> > On Sun, Mar 20, 2022 at 8:41 AM Amit Kapila wrote:
> >>
> >> On Fri, Mar 18, 2022 at 10:42 PM Tomas Vondra
> >> wrote:
> >>
> >>> So t
he existing row filter. It seems similar thing works
for column list as well ("alter publication pub1 set table t1 (c2)
where (c2 > 10)"). If I am not missing anything, I don't think we need
additional Alter Table syntax.
> So maybe we should just ditch ALTER
> TABLE entirely.
>
Yeah, I agree especially if my above understanding is correct.
--
With Regards,
Amit Kapila.
are slightly different for row filters and
column lists, so we need them (combine of filters) for one but not for
the other. Now, for the sake of consistency with row filters, we can
do it but as such there won't be any problem or maybe we can just add
a comment for the same in code.
--
With Regards,
Amit Kapila.
lder. I haven't
checked whether there is any need to reduce some tests but it seems
worth checking.
--
With Regards,
Amit Kapila.
On Tue, Mar 22, 2022 at 7:25 AM houzj.f...@fujitsu.com
wrote:
>
> > On Monday, March 21, 2022 6:01 PM Amit Kapila
> > wrote:
>
> Oh, sorry, I posted the wrong patch, here is the correct one.
>
The test change looks good to me. I think additionally we can verify
that the
l or not. Is it required to create a new relfilenode for
non-transactional messages? If not that could be costly?
--
With Regards,
Amit Kapila.
uld be
> removed as well.
>
As per my understanding, the removed feature is "Alter Publication ...
Alter Table ...". The example here "Alter Publication ... Set Table
.." should still work as mentioned in my email[1].
[1] -
https://www.postgresql.org/message-id/CAA4eK1L6YTcx%3DyJfdudr-y98Wcn4rWX4puHGAa2nxSCRb3fzQw%40mail.gmail.com
--
With Regards,
Amit Kapila.
h worrying about though -- it doesn't seem
>> like it could be hit more than once per query in normal cases.
>>
>> regards, tom lane
>
>
> Thanks Tom for replying.
>
> What do you think of the following patch ?
>
Your patch looks good
On Tue, Mar 22, 2022 at 5:41 PM Petr Jelinek
wrote:
>
> > On 22. 3. 2022, at 13:09, Amit Kapila wrote:
> >
> > On Mon, Mar 21, 2022 at 4:25 AM Tomas Vondra
> > wrote:
> >>
> >> Hi,
> >>
> >> Attached is a rebased patch, addressin
On Wed, Mar 23, 2022 at 9:30 AM Zhihong Yu wrote:
>
> On Tue, Mar 22, 2022 at 8:45 PM Amit Kapila wrote:
>>
>> On Tue, Mar 22, 2022 at 3:39 AM Zhihong Yu wrote:
>>
>> Your patch looks good to me. I have found one more similar instance in
>> the same file
On Thu, Mar 24, 2022 at 4:11 AM Tomas Vondra
wrote:
>
> On 3/21/22 12:28, Amit Kapila wrote:
> > On Fri, Mar 18, 2022 at 8:13 PM Tomas Vondra
> > wrote:
> >>
> >> Ah, thanks for reminding me - it's hard to keep track of all the issues
> >> in
his then I feel it is better to mark
this patch as Returned with feedback or Rejected. We can come back to
it later if we see more demand for this.
--
With Regards,
Amit Kapila.
On Tue, Dec 1, 2020 at 7:55 AM Ajin Cherian wrote:
>
> On Tue, Dec 1, 2020 at 12:46 AM Amit Kapila wrote:
>
>
>
> > Sure, but you can see in your example above it got skipped due to
> > start_decoding_at not due to DecodingContextReady. So, the problem as
> > m
On Mon, Nov 30, 2020 at 7:17 PM Amit Kapila wrote:
>
> On Mon, Nov 30, 2020 at 2:36 PM Ajin Cherian wrote:
>
> Sure, but you can see in your example above it got skipped due to
> start_decoding_at not due to DecodingContextReady. So, the problem as
> mentioned by me pre
On Tue, Dec 1, 2020 at 11:38 AM Dilip Kumar wrote:
>
> On Mon, Nov 30, 2020 at 6:49 PM Amit Kapila wrote:
> >
> >
> > What is going on here is that the expected streaming file is missing.
> > Normally, the first time we send a stream of changes (some percentage
accepted by this community.
>
Why not enhance test_decoding apart from mentioning in docs to explain
such concepts? It might help in some code-coverage as well depending
on what we want to cover.
[1] - https://www.postgresql.org/docs/devel/logical-replication.html
--
With Regards,
Amit Kapila.
On Wed, Dec 2, 2020 at 1:20 PM Dilip Kumar wrote:
>
> On Tue, Dec 1, 2020 at 11:38 AM Dilip Kumar wrote:
> >
> > On Mon, Nov 30, 2020 at 6:49 PM Amit Kapila wrote:
> > >
> > > On Mon, Nov 30, 2020 at 3:14 AM Noah Misch wrote:
> > > >
> &
On Wed, Dec 2, 2020 at 12:47 PM Ajin Cherian wrote:
>
> On Tue, Dec 1, 2020 at 6:26 PM Amit Kapila wrote:
> > > One idea could be that the subscriber skips the transaction if it sees
> > > the transaction is already prepared.
> > >
> >
> > To skip
ode.
I see that this code is added as part of commit
7c4f52409a8c7d85ed169bbbc1f6092274d03920 (Logical replication support
for initial data copy).
Thoughts?
[1] -
https://www.postgresql.org/message-id/cahut+puemk4so8ogzxc_ftzpkga8uc-y5qi-krqhsy_p0i3...@mail.gmail.com
--
With Regards,
Amit Ka
We start recording changes in ReorderBufferTXN even before we reach
SNAPBUILD_CONSISTENT state so that if the commit is encountered after
reaching that we should be able to send the changes of the entire
transaction. Now, while recording changes if the reorder buffer memory
has exceeded logical_dec
On Thu, Dec 3, 2020 at 7:04 PM Ashutosh Bapat
wrote:
>
> On Thu, Dec 3, 2020 at 2:55 PM Amit Kapila wrote:
> >
> > The tablesync worker in logical replication performs the table data
> > sync in a single transaction which means it will copy the initial data
> >
On Fri, Dec 4, 2020 at 7:53 AM Craig Ringer
wrote:
>
> On Thu, 3 Dec 2020 at 17:25, Amit Kapila wrote:
>
>
> > The reason why I am looking into this area is to support the logical
> > decoding of prepared transactions. See the problem [1] reported by
> > Peter Sm
On Fri, Dec 4, 2020 at 7:53 AM Craig Ringer
wrote:
>
> On Thu, 3 Dec 2020 at 17:25, Amit Kapila wrote:
>
> > Is there any fundamental problem if
> > we commit the transaction after initial copy and slot creation in
> > LogicalRepSyncTableStart and then allow the appl
On Fri, Dec 4, 2020 at 10:29 AM Amit Kapila wrote:
>
> On Fri, Dec 4, 2020 at 7:53 AM Craig Ringer
> wrote:
> >
> > On Thu, 3 Dec 2020 at 17:25, Amit Kapila wrote:
> >
> > > Is there any fundamental problem if
> > > we commit the transa
On Fri, Dec 4, 2020 at 10:29 AM Amit Kapila wrote:
>
> On Fri, Dec 4, 2020 at 7:53 AM Craig Ringer
> wrote:
> >
> > On Thu, 3 Dec 2020 at 17:25, Amit Kapila wrote:
> >
> > > Is there any fundamental problem if
> > > we commit the transa
On Fri, Dec 4, 2020 at 11:19 AM Dilip Kumar wrote:
>
> On Thu, Dec 3, 2020 at 5:33 PM Amit Kapila wrote:
> >
> > We start recording changes in ReorderBufferTXN even before we reach
> > SNAPBUILD_CONSISTENT state so that if the commit is encountered after
> > reac
t; relation just after a file extension failure. I think we are thinking
> that that doesn't happen during recovery. Although it seems to me
> true, I'm not confident.
>
Yeah, I also think it might not be worth depending upon whether smgr
close has been done before or not. I feel the current idea of using
'cached' parameter is relatively solid and we should rely on that.
Also, which means that in DropRelFileNodesAllBuffers() we should rely
on the same, I think doing things differently in this regard will lead
to confusion. I agree in some cases we might not get benefits but it
is more important to be correct and keep the code consistent to avoid
introducing bugs now or in the future.
--
With Regards,
Amit Kapila.
On Fri, Dec 4, 2020 at 7:12 PM Ashutosh Bapat
wrote:
>
> On Thu, Dec 3, 2020 at 7:24 PM Amit Kapila wrote:
> >
> > On Thu, Dec 3, 2020 at 7:04 PM Ashutosh Bapat
> > wrote:
> > >
> > > On Thu, Dec 3, 2020 at 2:55 PM Amit Kapila
> > > wro
On Fri, Dec 4, 2020 at 1:50 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
>
> > The earlier version of the patch having all these ideas
> > implemented is attached
> > (Infrastructure-to-execute-pending-undo-actions and
> > Provide-interfaces-to-store-and
look like:
Gather (actual time=970.524..972.913 rows=0 loops=1)
-> Create t1_test
Workers Planned: 2
Workers Launched: 2
-> Parallel Seq Scan on t1 (actual time=0.028..86.623 rows=33 loops=3)
I would prefer it to be:
Gather (actual time=970.524..972.913 rows=0 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Create t1_test
-> Parallel Seq Scan on t1 (actual time=0.028..86.623 rows=33 loops=3)
This way it looks like the writing part is done below the Gather node
and also it will match the Parallel Insert patch of Greg.
--
With Regards,
Amit Kapila.
On Mon, Dec 7, 2020 at 6:20 AM Craig Ringer
wrote:
>
> On Sat, 5 Dec 2020, 10:03 Amit Kapila, wrote:
>>
>> On Fri, Dec 4, 2020 at 7:12 PM Ashutosh Bapat
>> wrote:
>>
>> I think the problem is not that the changes are visible after COPY
>> rather it is
the
> resulting lag.
>
This is just for the initial tablesync phase. I think it is equivalent
to saying that during basebackup, we need to parallelly start physical
replication. I agree that sometimes it can take a lot of time to copy
large tables but it will be just one time and no worse than the other
situations like basebackup.
[1] -
https://www.postgresql.org/message-id/CAA4eK1KFsjf6x-S7b0dJLvEL3tcn9x-voBJiFoGsccyH5xgDzQ%40mail.gmail.com
--
With Regards,
Amit Kapila.
On Mon, Dec 7, 2020 at 9:21 AM Amit Kapila wrote:
>
> On Mon, Dec 7, 2020 at 6:20 AM Craig Ringer
> wrote:
> >
>
> >>
> >> I am not sure why but it seems acceptable to original authors that the
> >> data of transactions are visibly partially during
eam to multiple subscribers, saving on logical decoding and
> reorder buffering overheads and write-multiplication costs
>
I think doing parallel apply and ability to decode a txn once are
really good improvements independent of all the work you listed.
Thanks for sharing your knowledge.
--
With Regards,
Amit Kapila.
te table as], the
> query_level is always 1 because there is no subquery.
> So even if gather is not the top node, parallel cost will still be ignored.
>
> Is that works as expected ?
>
I don't think that is expected and is not the case without this patch.
The cost shouldn't be changed for existing cases where the write is
not pushed to workers.
--
With Regards,
Amit Kapila.
er if you can expand the use case a bit more. It
is not very clear from what you have written.
--
With Regards,
Amit Kapila.
On Mon, Dec 7, 2020 at 3:44 PM Bharath Rupireddy
wrote:
>
> On Mon, Dec 7, 2020 at 2:55 PM Amit Kapila wrote:
> >
> > On Mon, Dec 7, 2020 at 11:32 AM Hou, Zhijie
> > wrote:
> > >
> > > + if (!(root->parse->isForCTAS
On Mon, Dec 7, 2020 at 12:32 PM k.jami...@fujitsu.com
wrote:
>
> On Friday, December 4, 2020 8:27 PM, Amit Kapila
> wrote:
> > On Fri, Nov 27, 2020 at 11:36 AM Kyotaro Horiguchi
> > wrote:
> > >
> > > At Fri, 27 Nov 2020 02:19:57 +, "k.jami...
On Mon, Dec 7, 2020 at 4:20 PM Bharath Rupireddy
wrote:
>
> On Mon, Dec 7, 2020 at 4:04 PM Amit Kapila wrote:
> >
> > What is the need of checking query_level when 'isForCTAS' is set only
> > when Gather is a top-node?
> >
>
> isForCTAS is getting
On Tue, Dec 8, 2020 at 6:23 AM Kyotaro Horiguchi
wrote:
>
> At Tue, 08 Dec 2020 09:45:53 +0900 (JST), Kyotaro Horiguchi
> wrote in
> > At Mon, 7 Dec 2020 17:18:31 +0530, Amit Kapila
> > wrote in
> > > Hmm, how is it possible if Insert is done before Truncate? The
On Tue, Dec 8, 2020 at 7:24 AM Kyotaro Horiguchi
wrote:
>
> I'm out of it more than usual..
>
> At Tue, 08 Dec 2020 09:45:53 +0900 (JST), Kyotaro Horiguchi
> wrote in
> > At Mon, 7 Dec 2020 17:18:31 +0530, Amit Kapila
> > wrote in
> > > On Mon, Dec
On Mon, Dec 7, 2020 at 2:21 PM Amit Kapila wrote:
>
> On Mon, Dec 7, 2020 at 9:21 AM Amit Kapila wrote:
> >
> > On Mon, Dec 7, 2020 at 6:20 AM Craig Ringer
> > wrote:
> > >
> >
> > >>
> > >> I am not sure why but it seems acceptab
On Tue, Dec 8, 2020 at 10:41 AM Kyotaro Horiguchi
wrote:
>
> At Tue, 8 Dec 2020 08:08:25 +0530, Amit Kapila
> wrote in
> > On Tue, Dec 8, 2020 at 7:24 AM Kyotaro Horiguchi
> > wrote:
> > > We drop
> > > buffers for the old relfilenode on trunc
cyH5xgDzQ%40mail.gmail.com
[4] -
https://www.postgresql.org/message-id/CAA4eK1Ld9XaLoTZCoKF_gET7kc1fDf8CPR3CM48MQb1N1jDLYg%40mail.gmail.com
--
With Regards,
Amit Kapila.
think of a way to use an optimized path for such cases
but I don't agree with your comment on if it is common enough that we
leave this optimization entirely for the truncate path.
--
With Regards,
Amit Kapila.
in grouping_planner(), if the
query_level is 1, it is for CTAS, and it doesn't have a chance to
create UPPER_REL (doesn't have grouping, order, limit, etc clause)
then we can probably assume that the Gather will be top_node. I am not
sure about this but I think it is worth exploring.
--
With Regards,
Amit Kapila.
limit_needed(root->parse))
> > if (root->parse->sortClause)
> > if (root->parse->distinctClause)
> > if (root->parse->hasWindowFuncs)
> > if (root->parse->groupClause || root->parse->groupingSets ||
> > root->parse->hasAggs || root->root->hasHavingQual)
> >
>
> Thanks Amit and Hou. I will look into these areas and get back soon.
>
It might be better to split the patch for this such that in the base
patch, we won't consider anything special for gather costing w.r.t
CTAS and in the next patch, we consider all the checks discussed
above.
--
With Regards,
Amit Kapila.
tly on any big-endian system. I'm currently planning to
> drop the test suite changes from the commit, but I could keep them if folks
> like them. (They'd need more comments and timeout handling.)
>
I think it is better to keep this test which can always test multiple
streams on the subscriber.
Thanks for working on this.
--
With Regards,
Amit Kapila.
can now allow updates from the worker, but what
> is the problem if we allow the parallel select even if there is an
> update in the leader?
>
I think we can't allow update even in leader without having a
mechanism for a shared combocid table. Right now, we share the
ComboCids at the beginning of the parallel query and then never change
it during the parallel query but if we allow updates in the leader
backend which can generate a combocid then we need a mechanism to
propagate that change. Does this make sense?
--
With Regards,
Amit Kapila.
On Wed, Dec 9, 2020 at 6:32 AM Kyotaro Horiguchi
wrote:
>
> At Tue, 8 Dec 2020 16:28:41 +0530, Amit Kapila
> wrote in
> > On Tue, Dec 8, 2020 at 12:13 PM tsunakawa.ta...@fujitsu.com
> > wrote:
> > >
> > > From: Jamison, Kirk/ジャミソン カーク
> > > &
ey and probably other conditions if possible. It is better to
protect such cases from happening due to any bugs. Is there a reason
you have not handled it?
[1] -
https://www.postgresql.org/message-id/CAA4eK1KyftVDgovvRQmdV1b%3DnN0R-KqdWZqiu7jZ1GYQ7SO9OA%40mail.gmail.com
--
With Regards,
Amit Kapila.
On Wed, Dec 9, 2020 at 4:18 PM Dilip Kumar wrote:
>
> On Wed, Dec 9, 2020 at 4:03 PM Amit Kapila wrote:
> >
> > On Wed, Dec 9, 2020 at 2:38 PM Dilip Kumar wrote:
> > >
> > > On Wed, Dec 9, 2020 at 10:11 AM Greg Nancarrow
> > > wrote:
> > &
On Thu, Dec 10, 2020 at 7:11 AM Kyotaro Horiguchi
wrote:
>
> At Wed, 9 Dec 2020 16:27:30 +0530, Amit Kapila
> wrote in
> > On Wed, Dec 9, 2020 at 6:32 AM Kyotaro Horiguchi
> > > Mmm. At least btree doesn't need to call smgrnblocks except at
> > > expans
ptimization won't work like when during recovery we
have to just perform TRUNCATE.
--
With Regards,
Amit Kapila.
On Fri, Dec 11, 2020 at 5:54 AM k.jami...@fujitsu.com
wrote:
>
> On Thursday, December 10, 2020 8:12 PM, Amit Kapila wrote:
> > On Thu, Dec 10, 2020 at 1:40 PM k.jami...@fujitsu.com
> > wrote:
> > >
> > > Yes, I have tested that optimization works for index re
rderBufferProcessMessage(), but the
> later may break API compatibility.
>
+1 for the comment change but I am not sure if it is a good idea to
change the API name.
--
With Regards,
Amit Kapila.
t; > string keys have to exceed 8 or 16 bytes would be enough protection.
>
> Agreed. I expect (2) gives most of the benefit. Requiring 8-byte capacity
> should be harmless, and most architectures can zero 8 bytes in one
> instruction. Requiring more bytes trades specificity for sensitivity.
>
+1. I also think in most cases (2) would be sufficient to avoid such
bugs. Adding restriction on string size might annoy some out-of-core
user which is already using small strings. However, adding an 8-byte
restriction on string size would be still okay.
--
With Regards,
Amit Kapila.
e does the second slot refers to 'shared' or 'private'? It is not
very clear what you mean by "we are not doing reading from the
stream', do you mean to say that decoding happens in the slot but the
output plugin just throws away the streamed data and in the end just
send the feedback?
--
With Regards,
Amit Kapila.
On Mon, Dec 14, 2020 at 2:59 PM Amit Kapila wrote:
>
Today, I looked at one of the issues discussed earlier in this thread
[1] which is that decoding can block (or deadlock can happen) when the
user explicitly locks the catalog relation (like Lock pg_class) or
perform Cluster on non-relmap
rforming read/decode on the private slot) could be
the reason why it lagging behind. If you want to use as a reserve slot
then you probably want to at least perform
pg_replication_slot_advance() to move it to the required position. The
restart_lsn won't move unless you read/decode from that slot.
--
With Regards,
Amit Kapila.
On Tue, Dec 15, 2020 at 11:25 AM Ashutosh Bapat
wrote:
>
> On Mon, Dec 14, 2020 at 3:14 PM Amit Kapila wrote:
>>
>> On Mon, Dec 14, 2020 at 2:45 PM Ashutosh Bapat
>> wrote:
>> >
>> > The name of the function suggests that the given message will be queu
m the same publisher again by checking origin_lsn
> in TwoPhaseFileHeader I guess we can skip the PREPARE message only
> when the existing prepared transaction has the same LSN and the same
> identifier. To be exact, it’s still possible that the subscriber gets
> two PREPARE messages having the same LSN and name from two different
> publishers but it’s unlikely happen in practice.
>
The idea sounds reasonable. I'll try and see if this works.
Thanks.
--
With Regards,
Amit Kapila.
y committers take care of this. If you want to
> > remember it, maybe adding a line for it in the commit message should
> > be okay. For now, I have removed this from the patch.
> >
> >
> > --
> > With Regards,
> > Amit Kapila.
>
> I have reviewed the
On Thu, Dec 17, 2020 at 9:02 AM Amit Kapila wrote:
>
> On Thu, Dec 17, 2020 at 7:02 AM Ajin Cherian wrote:
> >
> >
> > I have reviewed the changes, did not have any new comments.
> > While testing, I found an issue in this patch. During initialisation,
> > th
On Thu, Dec 17, 2020 at 6:19 PM Amit Kapila wrote:
>
> On Wed, Dec 16, 2020 at 2:54 PM Amit Kapila wrote:
> >
> > On Wed, Dec 16, 2020 at 1:04 PM Masahiko Sawada
> > wrote:
> > >
> >
> > > Also, I guess we can improve the description of
> &
On Thu, Dec 17, 2020 at 9:30 AM Ajin Cherian wrote:
>
> On Thu, Dec 17, 2020 at 2:41 PM Amit Kapila wrote:
>
> > On again thinking about this, I think it is good to disable it during
> > slot initialization but will it create any problem because during slot
> > initial
; role is not filled and there are no objections.
>
I haven't seen anybody volunteering yet.
--
With Regards,
Amit Kapila.
you are trying to create this patch atop logical
decoding of 2PC patch but I think it is better to create this as an
independent patch and then use it to test 2PC problem. Also, please
explain what kind of testing you did to ensure that it works properly
after the table sync worker restarts after the crash.
--
With Regards,
Amit Kapila.
On Fri, Dec 18, 2020 at 3:37 PM Ashutosh Bapat
wrote:
>
> On Wed, Dec 16, 2020 at 8:00 AM Amit Kapila wrote:
>>
>> How about something like below:
>> A transactional message is queued to be processed upon commit and a
>> non-transactional message gets processed im
On Sat, Dec 19, 2020 at 12:10 PM Amit Kapila wrote:
>
> On Fri, Dec 18, 2020 at 6:41 PM Peter Smith wrote:
> >
>
> I understand why you are trying to create this patch atop logical
> decoding of 2PC patch but I think it is better to create this as an
> independent patch
r without it because there is nothing to scan in heap.
Additionally, we can have an Assert or elog in InitializeParallelDSM
to ensure that it is never invoked by parallel worker.
--
With Regards,
Amit Kapila.
On Mon, Dec 21, 2020 at 3:17 PM Peter Smith wrote:
>
> On Mon, Dec 21, 2020 at 4:23 PM Amit Kapila wrote:
>
> > Few other comments:
> > ==
>
> Thanks for your feedback.
>
> > 1.
> > * FIXME 3 - Crashed tablesync workers may also have
counted for the recent
write. But that should be fine because there must not be any buffers
after that file size.
XXX We would make the extra lseek call for the unoptimized paths but
that is okay because we do it just for the first fork and we anyway
have to scan the entire buffer pool the cost of which is so high that
the extra lseek call won't make any visible difference. However, we
can use InRecovery flag to avoid the additional cost but that doesn't
seem worth it."
Thoughts?
--
With Regards,
Amit Kapila.
On Tue, Dec 22, 2020 at 7:13 AM tsunakawa.ta...@fujitsu.com
wrote:
>
> From: Amit Kapila
> > This answers the second part of the question but what about the first
> > part (We hold a buffer partition lock, and have done a lookup in th
> > mapping table. Why ar
On Tue, Dec 22, 2020 at 8:18 AM tsunakawa.ta...@fujitsu.com
wrote:
>
> From: Amit Kapila
> > Why would all client backends wait for AccessExclusive lock on this
> > relation? Say, a client needs a buffer for some other relation and
> > that might evict this buffer afte
On Tue, Dec 22, 2020 at 8:12 AM Kyotaro Horiguchi
wrote:
>
> At Tue, 22 Dec 2020 08:08:10 +0530, Amit Kapila
> wrote in
>
> > Why would all client backends wait for AccessExclusive lock on this
> > relation? Say, a client needs a buffer for some other relation and
&
On Tue, Dec 22, 2020 at 8:30 AM Kyotaro Horiguchi
wrote:
>
> At Tue, 22 Dec 2020 02:48:22 +, "tsunakawa.ta...@fujitsu.com"
> wrote in
> > From: Amit Kapila
> > > Why would all client backends wait for AccessExclusive lock on this
> > > relation?
On Tue, Dec 22, 2020 at 2:51 PM Ajin Cherian wrote:
>
> On Sat, Dec 19, 2020 at 2:13 PM Amit Kapila wrote:
>
> > Okay, I have changed the rollback_prepare API as discussed above and
> > accordingly handle the case where rollback is received without prepare
> > in app
On Tue, Dec 22, 2020 at 2:55 PM Amit Kapila wrote:
>
> Apart from tests, do let me know if you are happy with the changes in
> the patch? Next, I'll look into DropRelFileNodesAllBuffers()
> optimization patch.
>
Review of v35-0004-Optimize-DropRelFileNodesAllBu
On Wed, Dec 23, 2020 at 6:30 AM k.jami...@fujitsu.com
wrote:
>
> On Tuesday, December 22, 2020 6:25 PM, Amit Kapila wrote:
>
> > Apart from tests, do let me know if you are happy with the changes in the
> > patch? Next, I'll look into DropRelFileNodesAllBuffers() optimiz
>
I don't think we support parallel scan for temporary tables. Can you
please try once both of these operations without Insert being
involved? If you are able to produce a parallel plan without Insert
then we can see why it is not supported with Insert.
--
With Regards,
Amit Kapila.
is not enabled when temporary table is
> in select part.
>
I think Select can be parallel for this case and we should support this case.
--
With Regards,
Amit Kapila.
On Tue, Dec 22, 2020 at 5:41 PM Amit Kapila wrote:
>
> On Tue, Dec 22, 2020 at 2:55 PM Amit Kapila wrote:
> >
> > Apart from tests, do let me know if you are happy with the changes in
> > the patch? Next, I'll look into DropRelFileNodesAllBuffers()
> > optimi
On Wed, Dec 23, 2020 at 1:07 PM k.jami...@fujitsu.com
wrote:
>
> On Tuesday, December 22, 2020 9:11 PM, Amit Kapila wrote:
>
> > In this code, I am slightly worried about the additional cost of each time
> > checking smgrexists. Consider a case where there are many relation
101 - 200 of 6713 matches
Mail list logo
