Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-13 Thread Bertrand Drouvot
Hi, On Tue, Mar 11, 2025 at 06:10:32AM +, Bertrand Drouvot wrote: > WalSndWaitForWal() is being used only for logical walsender. So we'd need to > find another location for the physical walsender case. One option is to keep > the > WalSndLoop() location and control the reports frequency. I

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-11 Thread Bertrand Drouvot
Hi, On Mon, Mar 10, 2025 at 09:23:50AM +0900, Michael Paquier wrote: > On Mon, Mar 03, 2025 at 11:54:39AM +, Bertrand Drouvot wrote: > > So it does not look like what we're adding here can be seen as a primary > > bottleneck > > but that is probably worth implementing the "have_iostats" optim

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-10 Thread Bertrand Drouvot
Hi, On Mon, Mar 10, 2025 at 02:52:42PM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Mar 10, 2025 at 09:23:50AM +0900, Michael Paquier wrote: > > On Mon, Mar 03, 2025 at 11:54:39AM +, Bertrand Drouvot wrote: > > > So it does not look like what we're adding here can be seen as a primary >

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-09 Thread Michael Paquier
On Mon, Mar 03, 2025 at 11:54:39AM +, Bertrand Drouvot wrote: > So it does not look like what we're adding here can be seen as a primary > bottleneck > but that is probably worth implementing the "have_iostats" optimization > attached. > > Also, while I did not measure any noticeable extra l

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-03 Thread Bertrand Drouvot
Hi, On Mon, Mar 03, 2025 at 10:51:19AM +0900, Michael Paquier wrote: > On Fri, Feb 28, 2025 at 10:39:31AM +, Bertrand Drouvot wrote: > > That sounds a good idea to measure the impact of those extra calls and see > > if we'd need to mitigate the impacts. I'll collect some data. So I did some t

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-02 Thread Michael Paquier
On Fri, Feb 28, 2025 at 10:39:31AM +, Bertrand Drouvot wrote: > That sounds a good idea to measure the impact of those extra calls and see > if we'd need to mitigate the impacts. I'll collect some data. Thanks. -- Michael signature.asc Description: PGP signature

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-28 Thread Bertrand Drouvot
Hi, On Fri, Feb 28, 2025 at 02:41:34PM +0900, Michael Paquier wrote: > On Wed, Feb 26, 2025 at 09:48:50AM +, Bertrand Drouvot wrote: > > Yeah I think that makes sense, done that way in the attached. > > > > Speaking about physical walsender, I moved the test to 001_stream_rep.pl > > instead

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-28 Thread Michael Paquier
On Fri, Feb 28, 2025 at 02:41:34PM +0900, Michael Paquier wrote: > With smaller records, the loop can become hotter, can't it? Also, > there can be a high number of WAL senders on a single node, and I've > heard of some customers with complex logical decoding deployments with > dozens of logical W

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-27 Thread Michael Paquier
On Wed, Feb 26, 2025 at 09:48:50AM +, Bertrand Drouvot wrote: > Yeah I think that makes sense, done that way in the attached. > > Speaking about physical walsender, I moved the test to 001_stream_rep.pl > instead > (would also fail without the fix). Hmm. I was doing some more checks with th

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-27 Thread Andres Freund
On 2025-02-27 12:09:46 +0900, Michael Paquier wrote: > I suspect that there would be cases where a single stats kind should be able > to handle both transactional and non-transactional flush cases. Splitting > that across two stats kinds would lead to a lot of duplication. Agreed. Table stats wil

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-26 Thread Bertrand Drouvot
Hi, On Wed, Feb 26, 2025 at 05:08:17AM -0500, Andres Freund wrote: > Hi, > > On 2025-02-26 15:37:10 +0900, Michael Paquier wrote: > > That's bad, worse for a logical WAL sender, because it means that we > > have no idea what kind of I/O happens in this process until it exits, > > and logical WAL

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-26 Thread Michael Paquier
On Wed, Feb 26, 2025 at 05:08:17AM -0500, Andres Freund wrote: > I think it's also bad that we don't have a solution for 1), even just for > normal connections. If a backend causes a lot of IO we might want to know > about that long before the longrunning transaction commits. > > I suspect the rig

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-26 Thread Andres Freund
Hi, On 2025-02-26 15:37:10 +0900, Michael Paquier wrote: > That's bad, worse for a logical WAL sender, because it means that we > have no idea what kind of I/O happens in this process until it exits, > and logical WAL senders could loop forever, since v16 where we've > begun tracking I/O. FWIW, I

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-26 Thread Bertrand Drouvot
Hi, On Wed, Feb 26, 2025 at 03:37:10PM +0900, Michael Paquier wrote: > On Tue, Feb 25, 2025 at 01:42:08PM +, Bertrand Drouvot wrote: > > Now we can see that the numbers increased for the relation object and that > > we > > get non zeros numbers for the wal object too (which makes fully sense)

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 01:42:08PM +, Bertrand Drouvot wrote: > Now we can see that the numbers increased for the relation object and that we > get non zeros numbers for the wal object too (which makes fully sense). > > With the attached patch applied, we would get the same numbers already in