On Mon, Apr 07, 2025 at 07:13:20AM +, Bertrand Drouvot wrote:
> As we now have 2 code paths I think we "really" need 2 tests. The attached
> (to apply on top of v7) seems to do the job.
Confirmed. I am sold on this extra location on HEAD, and it does not
impact the run time of the test as the
Hi,
On Mon, Apr 07, 2025 at 03:31:14PM +0900, Michael Paquier wrote:
> On Fri, Apr 04, 2025 at 09:33:46PM +0530, vignesh C wrote:
> > The new test added currently passes even without the patch. It would
> > be ideal to have a test that fails without the patch and passes once
> > the patch is appli
On Fri, Apr 04, 2025 at 09:33:46PM +0530, vignesh C wrote:
> The new test added currently passes even without the patch. It would
> be ideal to have a test that fails without the patch and passes once
> the patch is applied.
Right. The subscription test and logical WAL senders passes without
the
Hi,
On Tue, Mar 18, 2025 at 07:14:12PM +0800, Xuneng Zhou wrote:
> Hi,
>
> I performed some tests using elog
Thanks for the testing!
> (no so sure whether this is the proper way
> to do it) to monitor the new method.
Well that simple enough and that works well if the goal is just to "count" ;-
On Mon, 31 Mar 2025 at 18:35, Bertrand Drouvot
wrote:
>
> Hi,
>
> On Mon, Mar 24, 2025 at 08:41:20AM +0900, Michael Paquier wrote:
> > On Wed, Mar 19, 2025 at 04:00:49PM +0800, Xuneng Zhou wrote:
> > > Hi,
> > > Moving the other two provides a more complete view of the settings. For
> > > newcomer
On Fri, 4 Apr 2025 at 10:31, Bertrand Drouvot
wrote:
>
> Hi,
>
> On Thu, Apr 03, 2025 at 03:23:31PM +0530, vignesh C wrote:
> > Can we add it to one of the subscription tests, such as 001_rep_changes.pl?
>
> Yeah that sounds like a good place for it. Done in the attached.
The new test added curre
Hi,
On Thu, Apr 03, 2025 at 03:23:31PM +0530, vignesh C wrote:
> Can we add it to one of the subscription tests, such as 001_rep_changes.pl?
Yeah that sounds like a good place for it. Done in the attached.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amaz
On Thu, 3 Apr 2025 at 12:54, Bertrand Drouvot
wrote:
>
> Hi,
>
> On Thu, Apr 03, 2025 at 09:25:02AM +0530, vignesh C wrote:
> > On Mon, 31 Mar 2025 at 18:35, Bertrand Drouvot
> > wrote:
> > >
> > Couple of suggestions:
>
> Thanks for looking at it!
>
> > 1) I felt we can include a similar verific
Hi,
On Thu, Apr 03, 2025 at 09:25:02AM +0530, vignesh C wrote:
> On Mon, 31 Mar 2025 at 18:35, Bertrand Drouvot
> wrote:
> >
> Couple of suggestions:
Thanks for looking at it!
> 1) I felt we can include a similar verification for one of the logical
> replication tests too:
> +# Wait for the wal
Hi,
On Mon, Mar 24, 2025 at 08:41:20AM +0900, Michael Paquier wrote:
> On Wed, Mar 19, 2025 at 04:00:49PM +0800, Xuneng Zhou wrote:
> > Hi,
> > Moving the other two provides a more complete view of the settings. For
> > newcomers(like me) to the codebase, seeing all three related values in one
> >
On Wed, Mar 19, 2025 at 04:00:49PM +0800, Xuneng Zhou wrote:
> Hi,
> Moving the other two provides a more complete view of the settings. For
> newcomers(like me) to the codebase, seeing all three related values in one
> place helps avoid a narrow view of the settings.
>
> But I am not sure that I
Hi,
Moving the other two provides a more complete view of the settings. For
newcomers(like me) to the codebase, seeing all three related values in one
place helps avoid a narrow view of the settings.
But I am not sure that I understand the cons of this well.
Bertrand Drouvot 于2025年3月19日周三 13:50写
Hi,
On Wed, Mar 19, 2025 at 02:26:47PM +0900, Michael Paquier wrote:
> On Wed, Mar 19, 2025 at 09:53:16AM +0800, Xuneng Zhou wrote:
> > I think these two conditions are good too. In a busy system, they are met
> > frequently, so the flush routine will be executed at least once every
> > second. Co
On Wed, Mar 19, 2025 at 09:53:16AM +0800, Xuneng Zhou wrote:
> I think these two conditions are good too. In a busy system, they are met
> frequently, so the flush routine will be executed at least once every
> second. Conversely, when WAL generation is low, there's simply less data to
> record, an
Hi,
On Wed, Mar 19, 2025 at 08:19:42AM +0900, Michael Paquier wrote:
>
> Done that part for now.
Thanks!
> It will be possible to look at the bug fix
> after the release freeze as it impacts stable branches as well.
Yes, let's do that. Adding a reminder on my side.
Regards,
--
Bertrand Drou
Hi,
I am OK with PGSTAT_MIN_INTERVAL. It has been used for flushing other
statistics as well. And monitoring systems are generally configured to poll
at one-second or longer intervals.
>
> I think that reporting at PGSTAT_MIN_INTERVAL is fine and more than
> enough. I
> mean, I 'm not sure that t
On Thu, Mar 13, 2025 at 01:18:45PM +, Bertrand Drouvot wrote:
> This particular sub-patch needs a rebase though, done in the attached. 0001
> remains unchanged as compared to the v4 one just shared up-thread. If 0001
> goes
> in, merging 0002 would be less beneficial (as compared to v3).
PgSt
On Tue, Mar 18, 2025 at 09:51:14AM +, Bertrand Drouvot wrote:
> Thanks!
Done that part for now. It will be possible to look at the bug fix
after the release freeze as it impacts stable branches as well.
--
Michael
signature.asc
Description: PGP signature
Hi,
I performed some tests using elog(no so sure whether this is the proper way
to do it) to monitor the new method. Here are the findings:
• With PGSTAT_MIN_INTERVAL set to 1000ms, the number of flush reports was
reduced to approximately 40–50 during the installcheck test suite.
• With PGSTAT_I
Hi,
On Tue, Mar 18, 2025 at 05:11:02PM +0900, Michael Paquier wrote:
> On Thu, Mar 13, 2025 at 01:18:45PM +, Bertrand Drouvot wrote:
> > This particular sub-patch needs a rebase though, done in the attached. 0001
> > remains unchanged as compared to the v4 one just shared up-thread. If 0001
>
Hi,
On Tue, Mar 11, 2025 at 06:10:32AM +, Bertrand Drouvot wrote:
> WalSndWaitForWal() is being used only for logical walsender. So we'd need to
> find another location for the physical walsender case. One option is to keep
> the
> WalSndLoop() location and control the reports frequency.
I
Hi,
On Thu, Mar 13, 2025 at 07:35:24PM +0800, Xuneng Zhou wrote:
> Hi,
> Given that PGSTAT_MIN_INTERVAL is the go-to setting for stats flushes
> * Unless called with 'force', pending stats updates are flushed happen once
> * per PGSTAT_MIN_INTERVAL (1000ms). When not forced, stats flushes do not
Hi,
On Thu, Mar 13, 2025 at 07:33:24PM +0800, Xuneng Zhou wrote:
> Regarding patch 0001, the optimization in pgstat_backend_have_pending_cb
> looks good:
Thanks for looking at it!
> bool
> pgstat_backend_have_pending_cb(void)
> {
> - return (!pg_memory_is_all_zeros(&PendingBackendStats,
> - size
Sorry, forgot to cc this one too.
-- Forwarded message -
发件人： Xuneng Zhou
Date: 2025年3月13日周四 19:29
Subject: Re: [BUG]: the walsender does not update its IO statistics until
it exits
To: Bertrand Drouvot
Hi,
Given that PGSTAT_MIN_INTERVAL is the go-to setting for stats flushes
Forgot to cc...
-- Forwarded message -
发件人： Xuneng Zhou
Date: 2025年3月13日周四 19:15
Subject: Re: [BUG]: the walsender does not update its IO statistics until
it exits
To: Bertrand Drouvot
Hi,
Thanks for working on this! I'm glad to see that the patch (
https://www.postgresq
Hi,
On Mon, Mar 10, 2025 at 09:23:50AM +0900, Michael Paquier wrote:
> On Mon, Mar 03, 2025 at 11:54:39AM +, Bertrand Drouvot wrote:
> > So it does not look like what we're adding here can be seen as a primary
> > bottleneck
> > but that is probably worth implementing the "have_iostats" optim
Hi,
On Mon, Mar 10, 2025 at 02:52:42PM +, Bertrand Drouvot wrote:
> Hi,
>
> On Mon, Mar 10, 2025 at 09:23:50AM +0900, Michael Paquier wrote:
> > On Mon, Mar 03, 2025 at 11:54:39AM +, Bertrand Drouvot wrote:
> > > So it does not look like what we're adding here can be seen as a primary
>
On Mon, Mar 03, 2025 at 11:54:39AM +, Bertrand Drouvot wrote:
> So it does not look like what we're adding here can be seen as a primary
> bottleneck
> but that is probably worth implementing the "have_iostats" optimization
> attached.
>
> Also, while I did not measure any noticeable extra l
Hi,
On Mon, Mar 03, 2025 at 10:51:19AM +0900, Michael Paquier wrote:
> On Fri, Feb 28, 2025 at 10:39:31AM +, Bertrand Drouvot wrote:
> > That sounds a good idea to measure the impact of those extra calls and see
> > if we'd need to mitigate the impacts. I'll collect some data.
So I did some t
On Fri, Feb 28, 2025 at 10:39:31AM +, Bertrand Drouvot wrote:
> That sounds a good idea to measure the impact of those extra calls and see
> if we'd need to mitigate the impacts. I'll collect some data.
Thanks.
--
Michael
signature.asc
Description: PGP signature
Hi,
On Fri, Feb 28, 2025 at 02:41:34PM +0900, Michael Paquier wrote:
> On Wed, Feb 26, 2025 at 09:48:50AM +, Bertrand Drouvot wrote:
> > Yeah I think that makes sense, done that way in the attached.
> >
> > Speaking about physical walsender, I moved the test to 001_stream_rep.pl
> > instead
On Fri, Feb 28, 2025 at 02:41:34PM +0900, Michael Paquier wrote:
> With smaller records, the loop can become hotter, can't it? Also,
> there can be a high number of WAL senders on a single node, and I've
> heard of some customers with complex logical decoding deployments with
> dozens of logical W
On Wed, Feb 26, 2025 at 09:48:50AM +, Bertrand Drouvot wrote:
> Yeah I think that makes sense, done that way in the attached.
>
> Speaking about physical walsender, I moved the test to 001_stream_rep.pl
> instead
> (would also fail without the fix).
Hmm. I was doing some more checks with th
On 2025-02-27 12:09:46 +0900, Michael Paquier wrote:
> I suspect that there would be cases where a single stats kind should be able
> to handle both transactional and non-transactional flush cases. Splitting
> that across two stats kinds would lead to a lot of duplication.
Agreed. Table stats wil
Hi,
On Wed, Feb 26, 2025 at 05:08:17AM -0500, Andres Freund wrote:
> Hi,
>
> On 2025-02-26 15:37:10 +0900, Michael Paquier wrote:
> > That's bad, worse for a logical WAL sender, because it means that we
> > have no idea what kind of I/O happens in this process until it exits,
> > and logical WAL
On Wed, Feb 26, 2025 at 05:08:17AM -0500, Andres Freund wrote:
> I think it's also bad that we don't have a solution for 1), even just for
> normal connections. If a backend causes a lot of IO we might want to know
> about that long before the longrunning transaction commits.
>
> I suspect the rig
Hi,
On 2025-02-26 15:37:10 +0900, Michael Paquier wrote:
> That's bad, worse for a logical WAL sender, because it means that we
> have no idea what kind of I/O happens in this process until it exits,
> and logical WAL senders could loop forever, since v16 where we've
> begun tracking I/O.
FWIW, I
Hi,
On Wed, Feb 26, 2025 at 03:37:10PM +0900, Michael Paquier wrote:
> On Tue, Feb 25, 2025 at 01:42:08PM +, Bertrand Drouvot wrote:
> > Now we can see that the numbers increased for the relation object and that
> > we
> > get non zeros numbers for the wal object too (which makes fully sense)
On Tue, Feb 25, 2025 at 01:42:08PM +, Bertrand Drouvot wrote:
> Now we can see that the numbers increased for the relation object and that we
> get non zeros numbers for the wal object too (which makes fully sense).
>
> With the attached patch applied, we would get the same numbers already in
Hi hackers,
while doing some tests for [1], I observed that $SUBJECT.
To observe this behavior on master:
1. create a logical replication slot
postgres=# SELECT * FROM pg_create_logical_replication_slot('logical_slot',
'test_decoding', false, true);
slot_name |lsn
--+--
40 matches
Mail list logo
