On Mon, Apr 26, 2021 at 7:05 AM Peter Smith wrote:
>
> Hi,
>
> While testing another WIP patch [1] a clashing GID problem was found,
> which gives us apply worker errors like:
>
> 2021-04-26 10:07:12.883 AEST [22055] ERROR: transaction identifier
> "pg_gid_16403_608" is already in use
> 2021-04-2
Hi,
While testing another WIP patch [1] a clashing GID problem was found,
which gives us apply worker errors like:
2021-04-26 10:07:12.883 AEST [22055] ERROR: transaction identifier
"pg_gid_16403_608" is already in use
2021-04-26 10:08:05.149 AEST [22124] ERROR: transaction identifier
"pg_gid_1
On Wed, Mar 31, 2021 at 7:20 PM Markus Wanner
wrote:
>
> On 31.03.21 15:18, Amit Kapila wrote:
> > On Wed, Mar 31, 2021 at 11:55 AM Markus Wanner
> >> The last sentences there now seems to relate to just the setting of
> >> "concurrent_abort", rather than the whole reason to invoke the
> >> prepar
On 31.03.21 15:18, Amit Kapila wrote:
On Wed, Mar 31, 2021 at 11:55 AM Markus Wanner
The last sentences there now seems to relate to just the setting of
"concurrent_abort", rather than the whole reason to invoke the
prepare_cb. And the reference to the "gid" is a bit lost. Maybe:
"Thus e
On Wed, Mar 31, 2021 at 11:55 AM Markus Wanner
wrote:
>
> On 31.03.21 06:39, Amit Kapila wrote:
> > I have slightly adjusted the comments, docs, and commit message. What
> > do you think about the attached?
>
> Thank you both, Amit and Ajin. This looks good to me.
>
> Only one minor gripe:
>
> >
On Wed, Mar 31, 2021 at 5:25 PM Markus Wanner <
markus.wan...@enterprisedb.com> wrote:
>
> The last sentences there now seems to relate to just the setting of
> "concurrent_abort", rather than the whole reason to invoke the
> prepare_cb. And the reference to the "gid" is a bit lost. Maybe:
>
>
On 31.03.21 06:39, Amit Kapila wrote:
I have slightly adjusted the comments, docs, and commit message. What
do you think about the attached?
Thank you both, Amit and Ajin. This looks good to me.
Only one minor gripe:
+ a prepared transaction with incomplete changes, in which case the
+
On Wed, Mar 31, 2021 at 6:42 AM Ajin Cherian wrote:
>
> Updated.
>
I have slightly adjusted the comments, docs, and commit message. What
do you think about the attached?
--
With Regards,
Amit Kapila.
v4-0001-Ensure-to-send-a-prepare-after-we-detect-concurre.patch
Description: Binary data
On Tue, Mar 30, 2021 at 10:29 PM Markus Wanner wrote:
>
> I just noticed as of PG13, concurrent_abort is part of ReorderBufferTXN,
> so it seems the prepare_cb (or stream_prepare_cb) can actually figure a
> concurrent abort happened (and the transaction may be incomplete).
> That's good and indee
On 30.03.21 11:54, Markus Wanner wrote:
I would recommend this more explicit API and communication over hiding
the concurrent abort in a prepare callback.
I figured we already have the ReorderBufferTXN's concurrent_abort flag,
thus I agree the prepare_cb is sufficient and revoke this recommend
On 30.03.21 11:12, Ajin Cherian wrote:
I found some documentation that already was talking about concurrent
aborts and updated that.
Thanks.
I just noticed as of PG13, concurrent_abort is part of ReorderBufferTXN,
so it seems the prepare_cb (or stream_prepare_cb) can actually figure a
concur
On 30.03.21 11:02, Amit Kapila wrote:
On Tue, Mar 30, 2021 at 12:00 PM Markus Wanner
Yes, this replaces the PREPARE I would do from the concurrent_abort
callback in a direct call to rb->prepare.
Because concurrent_abort()
internally trying to prepare transaction seems a bit ugly and not only
t
On Tue, Mar 30, 2021 at 7:10 PM Markus Wanner <
markus.wan...@enterprisedb.com> wrote:
> On 30.03.21 09:39, Ajin Cherian wrote:
> > Where do you suggest this be documented? From an externally visible
> > point of view, I dont see much of a surprise.
>
>
>
> So I suggest to document this as a cavea
On Tue, Mar 30, 2021 at 12:00 PM Markus Wanner
wrote:
>
> Hello Ajin,
>
> On 30.03.21 06:48, Ajin Cherian wrote:
> > For now, I've created a patch that addresses the problem reported using
> > the existing callbacks.
>
> Thanks.
>
> > Do have a look if this fixes the problem reported.
>
> Yes, thi
On 30.03.21 09:39, Ajin Cherian wrote:
Where do you suggest this be documented? From an externally visible
point of view, I dont see much of a surprise.
If you start to think about the option of committing a prepared
transaction from a different node, the danger becomes immediately
apparent:
On Tue, Mar 30, 2021 at 5:30 PM Markus Wanner <
markus.wan...@enterprisedb.com> wrote:
> Hello Ajin,
>
> On 30.03.21 06:48, Ajin Cherian wrote:
> > For now, I've created a patch that addresses the problem reported using
> > the existing callbacks.
>
> Thanks.
>
> > Do have a look if this fixes the
Hello Ajin,
On 30.03.21 06:48, Ajin Cherian wrote:
For now, I've created a patch that addresses the problem reported using
the existing callbacks.
Thanks.
Do have a look if this fixes the problem reported.
Yes, this replaces the PREPARE I would do from the concurrent_abort
callback in a d
On Mon, Mar 29, 2021 at 10:09 PM Markus Wanner <
markus.wan...@enterprisedb.com> wrote:
> On 29.03.21 13:02, Ajin Cherian wrote:
> > Nice catch, Markus.
> > Interesting suggestion Amit. Let me try and code this.
>
> Thanks, Ajin. Please consider this concurrent_abort callback as well.
> I think i
On 29.03.21 13:02, Ajin Cherian wrote:
Nice catch, Markus.
Interesting suggestion Amit. Let me try and code this.
Thanks, Ajin. Please consider this concurrent_abort callback as well.
I think it provides more flexibility for the output plugin and I would
therefore prefer it over a solution t
On Mon, Mar 29, 2021 at 8:33 PM Amit Kapila wrote:
> On Mon, Mar 29, 2021 at 12:36 PM Markus Wanner
> wrote:
> >
> > On 27.03.21 07:37, Amit Kapila wrote:
> > > Isn't it better to send prepare from the publisher in such a case so
> > > that subscribers can know about it when rollback prepared ar
On 29.03.21 11:33, Amit Kapila wrote:
You don't need an additional callback for that if we do what I am
suggesting above.
Ah, are you suggesting a different change, then? To make two-phase
transactions always send PREPARE even if concurrently aborted? In that
case, sorry, I misunderstood.
On Mon, Mar 29, 2021 at 12:36 PM Markus Wanner
wrote:
>
> On 27.03.21 07:37, Amit Kapila wrote:
> > Isn't it better to send prepare from the publisher in such a case so
> > that subscribers can know about it when rollback prepared arrives?
>
> That's exactly what this callback allows (among other
On 27.03.21 07:37, Amit Kapila wrote:
Isn't it better to send prepare from the publisher in such a case so
that subscribers can know about it when rollback prepared arrives?
That's exactly what this callback allows (among other options). It
provides a way for the output plugin to react to a t
On Fri, Mar 26, 2021 at 5:50 PM Markus Wanner
wrote:
>
> On 26.03.21 11:19, Amit Kapila wrote:
> > No, I am not assuming that. I am just trying to describe you that it
> > is not necessary that we will be able to detect concurrent abort in
> > each and every case.
>
> Sure. Nor am I claiming that
On 26.03.21 11:19, Amit Kapila wrote:
No, I am not assuming that. I am just trying to describe you that it
is not necessary that we will be able to detect concurrent abort in
each and every case.
Sure. Nor am I claiming that would be necessary or that the patch
changed anything about it.
As
On Fri, Mar 26, 2021 at 2:42 PM Markus Wanner
wrote:
>
> On 26.03.21 04:28, Amit Kapila wrote:
> > I think as you have noted that stream_abort or rollback_prepared will
> > be sent (the remaining changes in-between will be skipped) as we
> > decode them from WAL
>
> Yes, but as outlined, too late.
On 26.03.21 04:28, Amit Kapila wrote:
I think as you have noted that stream_abort or rollback_prepared will
be sent (the remaining changes in-between will be skipped) as we
decode them from WAL
Yes, but as outlined, too late. Multiple other transactions may get
decoded until the decoder reach
On Thu, Mar 25, 2021 at 2:37 PM Markus Wanner
wrote:
>
> here is another tidbit from our experience with using logical decoding.
> The attached patch adds a callback to notify the output plugin of a
> concurrent abort. I'll continue to describe the problem in more detail
> and how this additional
On 25.03.21 21:21, Andres Freund wrote:
... the code added as part of 7259736a6e5b7c75 ...
That's the streaming part, which can be thought of as a more general
variant of the two-phase decoding in that it allows multiple "flush
points" (invoking ReorderBufferProcessTXN). Unlike the PREPARE o
Hi,
On 2021-03-25 10:07:28 +0100, Markus Wanner wrote:
> This leads to the possibility of the transaction getting aborted concurrent
> to logical decoding. In that case, it is likely for the decoder to error on
> a catalog scan that conflicts with the abort of the transaction. The
> reorderbuffe
30 matches
Mail list logo
