Re: pg_logical_emit_message() misses a XLogFlush()

2023-10-17 Thread Michael Paquier
On Tue, Oct 17, 2023 at 11:57:33AM +0530, Amit Kapila wrote: > LGTM. Thanks, I've applied that, then. -- Michael signature.asc Description: PGP signature

Re: pg_logical_emit_message() misses a XLogFlush()

2023-10-16 Thread Amit Kapila
On Mon, Oct 16, 2023 at 12:47 PM Michael Paquier wrote: > > On Fri, Oct 13, 2023 at 03:20:30PM +0530, Amit Kapila wrote: > > I would prefer to associate the new parameter 'flush' with > > non-transactional messages as per the proposed patch. > > Check. > > > Is there a reason to make the functions

Re: pg_logical_emit_message() misses a XLogFlush()

2023-10-16 Thread Michael Paquier
On Fri, Oct 13, 2023 at 03:20:30PM +0530, Amit Kapila wrote: > I would prefer to associate the new parameter 'flush' with > non-transactional messages as per the proposed patch. Check. > Is there a reason to make the functions strict now when they were not earlier? These two are already STRICT o

Re: pg_logical_emit_message() misses a XLogFlush()

2023-10-13 Thread Amit Kapila
On Mon, Sep 11, 2023 at 5:13 PM Michael Paquier wrote: > > I'll need a bit more input from Fujii-san before doing anything about > his comments, still it looks like a doc issue to me that may need a > backpatch to clarify how the non-transactional case behaves. > I would prefer to associate the n

Re: pg_logical_emit_message() misses a XLogFlush()

2023-09-11 Thread Michael Paquier
On Mon, Sep 11, 2023 at 02:42:16PM +0900, bt23nguyent wrote: > A minor issue with the description here is that while the description for > the new flush argument in pg_logical_emit_message() with bytea type is > clearly declared, there is no description of flush argument in the former > pg_logical_

Re: pg_logical_emit_message() misses a XLogFlush()

2023-09-10 Thread bt23nguyent
On 2023-09-11 14:02, Michael Paquier wrote: On Mon, Sep 11, 2023 at 12:54:11PM +0900, Fujii Masao wrote: However, in the current patch, if "transactional" is set to false and "flush" is true, the function flushes the WAL immediately without considering synchronous_commit. Is this the intended be

Re: pg_logical_emit_message() misses a XLogFlush()

2023-09-10 Thread Michael Paquier
On Mon, Sep 11, 2023 at 12:54:11PM +0900, Fujii Masao wrote: > However, in the current patch, if "transactional" is set to false and > "flush" is true, the function flushes the WAL immediately without > considering synchronous_commit. Is this the intended behavior? > I'm not sure how the function s

Re: pg_logical_emit_message() misses a XLogFlush()

2023-09-10 Thread Fujii Masao
On 2023/08/16 16:51, Michael Paquier wrote: Anyway, attached is a patch to add a 4th argument "flush" that defaults to false. Thoughts about this version are welcome. When the "transactional" option is set to true, WAL including the record generated by the pg_logical_emit_message() function

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-16 Thread Michael Paquier
On Wed, Aug 16, 2023 at 12:01:01PM +0200, Tomas Vondra wrote: > To me losing messages seems like a bad thing, but if the users are aware > of it and are fine with it ... I'm simply arguing that if we conclude > this is a durability bug, we should not leave it unfixed because it > might have perform

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-16 Thread Tomas Vondra
On 8/16/23 06:13, Andres Freund wrote: > Hi, > > On 2023-08-16 03:20:53 +0200, Tomas Vondra wrote: >> On 8/16/23 02:33, Andres Freund wrote: >>> Hi, >>> >>> On 2023-08-16 06:58:56 +0900, Michael Paquier wrote: On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: > Shouldn't th

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-16 Thread Michael Paquier
On Tue, Aug 15, 2023 at 09:16:53PM -0700, Andres Freund wrote: > To be clear: I don't just object to backpatching, I also object to making > existing invocations flush WAL in HEAD. I do not at all object to adding a > parameter that indicates flushing, or a separate function to do so. The latter >

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Andres Freund
Hi, On 2023-08-16 12:37:21 +0900, Michael Paquier wrote: > I won't fight much if there are objections to backpatching, but that's > not really wise (no idea how much EDB's close flavor of BDR relies on > that). To be clear: I don't just object to backpatching, I also object to making existing inv

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Andres Freund
Hi, On 2023-08-16 03:20:53 +0200, Tomas Vondra wrote: > On 8/16/23 02:33, Andres Freund wrote: > > Hi, > > > > On 2023-08-16 06:58:56 +0900, Michael Paquier wrote: > >> On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: > >>> Shouldn't the flush be done only for non-transactional messag

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Michael Paquier
On Wed, Aug 16, 2023 at 03:20:53AM +0200, Tomas Vondra wrote: > So are you objecting to adding the flush in general, or just to the > backpatching part? > > IMHO we either guarantee durability of non-transactional messages, in > which case this would be a clear bug - and I'd say a fairly serious o

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Tomas Vondra
On 8/16/23 02:33, Andres Freund wrote: > Hi, > > On 2023-08-16 06:58:56 +0900, Michael Paquier wrote: >> On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: >>> Shouldn't the flush be done only for non-transactional messages? The >>> transactional case will be flushed by regular commit f

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Andres Freund
Hi, On 2023-08-16 06:58:56 +0900, Michael Paquier wrote: > On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: > > Shouldn't the flush be done only for non-transactional messages? The > > transactional case will be flushed by regular commit flush. > > Indeed, that would be better. I am

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Michael Paquier
On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: > Shouldn't the flush be done only for non-transactional messages? The > transactional case will be flushed by regular commit flush. Indeed, that would be better. I am sending an updated patch. I'd like to backpatch that, would there

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Tomas Vondra
On 8/15/23 08:38, Michael Paquier wrote: > Hi all, > > While playing with pg_logical_emit_message() and WAL replay, I have > noticed that LogLogicalMessage() inserts a record but forgets to make > sure that the record has been flushed. So, for example, if the system > crashes the message inser