Hi,
On 2022-02-25 11:32:24 -0800, Andres Freund wrote:
> On 2022-02-25 16:25:01 -0300, Euler Taveira wrote:
> > On Fri, Feb 25, 2022, at 11:52 AM, Greg Stark wrote:
> > > On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote:
> > > >
> > > > Hi,
> > > >
> > > > I was looking the shared memory stats p
On Wed, Mar 2, 2022 at 1:26 PM Andres Freund wrote:
>
> > > While rebasing, I was wondering why pgstat_reset_subscription_counter()
> > > has
> > > "all subscription counters" support. We don't have a function to reset all
> > > function stats or such either.
> > >
> >
> > We have similar thing f
Hi,
On 2022-03-02 12:39:57 +0530, Amit Kapila wrote:
> On Wed, Mar 2, 2022 at 10:39 AM Andres Freund wrote:
> >
> > Working on rebasing shared memory stats over this. Feels *much* better so
> > far.
> >
>
> Good to hear that it helps. BTW, there is another patch [1] that
> extends this view. I
On Wed, Mar 2, 2022 at 10:39 AM Andres Freund wrote:
>
> Working on rebasing shared memory stats over this. Feels *much* better so far.
>
Good to hear that it helps. BTW, there is another patch [1] that
extends this view. I think that patch is still not ready but once it
is ready (which I expect
Hi,
On 2022-03-02 07:35:46 +0530, Amit Kapila wrote:
> Pushed.
Thanks!
Working on rebasing shared memory stats over this. Feels *much* better so far.
While rebasing, I was wondering why pgstat_reset_subscription_counter() has
"all subscription counters" support. We don't have a function to rese
On Mon, Feb 28, 2022 at 1:15 PM tanghy.f...@fujitsu.com
wrote:
>
> On Mon, Feb 28, 2022 12:32 PM Amit Kapila wrote:
> >
> > I decided to keep this part of the docs as it is and fixed a few other
> > minor comments raised by you and Peter. Additionally, I have bumped
> > the PGSTAT_FILE_FORMAT_ID.
On Mon, Feb 28, 2022 12:32 PM Amit Kapila wrote:
>
> On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada
> wrote:
> >
> > On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila
> wrote:
> > >
> > > >
> > > > (2) doc/src/sgml/monitoring.sgml
> > > >
> > > > +Resets statistics for a single subscription
On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada wrote:
>
> On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila wrote:
> >
> > >
> > > (2) doc/src/sgml/monitoring.sgml
> > >
> > > +Resets statistics for a single subscription shown in the
> > > +pg_stat_subscription_stats view to
> > > ze
On Monday, February 28, 2022 12:57 PM Amit Kapila
> On Mon, Feb 28, 2022 at 8:49 AM osumi.takami...@fujitsu.com
> wrote:
> >
> > On Monday, February 28, 2022 11:34 AM Amit Kapila
> wrote:
> > > On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com
> > > wrote:
> > > >
> > > > On Saturday,
On Mon, Feb 28, 2022 at 8:49 AM osumi.takami...@fujitsu.com
wrote:
>
> On Monday, February 28, 2022 11:34 AM Amit Kapila
> wrote:
> > On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com
> > wrote:
> > >
> > > On Saturday, February 26, 2022 11:51 AM Amit Kapila
> > wrote:
> > > > I have
On Mon, Feb 28, 2022 at 8:59 AM Peter Smith wrote:
>
>
> 2. doc/src/sgml/monitoring.sgml
>
> +Resets statistics for a single subscription shown in the
> +pg_stat_subscription_stats view to zero. If
> +the argument is NULL, reset statistics for all
> +subscriptions.
Below are my comments for the v4 patch.
These are only nitpicking comments now; Otherwise, it LGTM.
(Sorry, now I see there are some overlaps with comments posted in the
last 20 mins so take or leave these as you wish)
==
1. doc/src/sgml/monitoring.sgml
-
- OID of the relation
On Monday, February 28, 2022 11:34 AM Amit Kapila
wrote:
> On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com
> wrote:
> >
> > On Saturday, February 26, 2022 11:51 AM Amit Kapila
> wrote:
> > > I have reviewed the latest version and made a few changes along with
> > > fixing some of th
On Mon, Feb 28, 2022 at 11:52 AM Amit Kapila wrote:
>
> On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada wrote:
> >
> > On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila
> > wrote:
> > >
> > > >
> > > > (2) doc/src/sgml/monitoring.sgml
> > > >
> > > > +Resets statistics for a single subscript
On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada wrote:
>
> On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila wrote:
> >
> > >
> > > (2) doc/src/sgml/monitoring.sgml
> > >
> > > +Resets statistics for a single subscription shown in the
> > > +pg_stat_subscription_stats view to
> > > ze
On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila wrote:
>
> On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com
> wrote:
> >
> > On Saturday, February 26, 2022 11:51 AM Amit Kapila
> > wrote:
> > > I have reviewed the latest version and made a few changes along with
> > > fixing
> > > some
On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com
wrote:
>
> On Saturday, February 26, 2022 11:51 AM Amit Kapila
> wrote:
> > I have reviewed the latest version and made a few changes along with fixing
> > some of the pending comments by Peter Smith. The changes are as
> > follows: (a)
On Sat, Feb 26, 2022 at 11:51 AM Amit Kapila wrote:
>
> On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada wrote:
> >
>
> I have reviewed the latest version and made a few changes along with
> fixing some of the pending comments by Peter Smith.
Thank you for updating the patch!
> The changes are a
On Saturday, February 26, 2022 11:51 AM Amit Kapila
wrote:
> I have reviewed the latest version and made a few changes along with fixing
> some of the pending comments by Peter Smith. The changes are as
> follows: (a) Removed m_databaseid in PgStat_MsgSubscriptionError as that is
> not required n
On Fri, Feb 25, 2022 at 7:26 AM Peter Smith wrote:
>
> Below are my review comments for the v3 patch.
>
...
> 2. doc/src/sgml/monitoring.sgml
>
> +
> pg_stat_subscription_activitypg_stat_subscription_activity
> + One row per subscription, showing statistics about subscription
> + a
On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada wrote:
>
I have reviewed the latest version and made a few changes along with
fixing some of the pending comments by Peter Smith. The changes are as
follows: (a) Removed m_databaseid in PgStat_MsgSubscriptionError as
that is not required now; (b) ch
Hi,
On 2022-02-25 16:25:01 -0300, Euler Taveira wrote:
> On Fri, Feb 25, 2022, at 11:52 AM, Greg Stark wrote:
> > On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote:
> > >
> > > Hi,
> > >
> > > I was looking the shared memory stats patch again.
> >
> > Can you point me to this thread? I looked fo
On Fri, Feb 25, 2022, at 11:52 AM, Greg Stark wrote:
> On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote:
> >
> > Hi,
> >
> > I was looking the shared memory stats patch again.
>
> Can you point me to this thread? I looked for it but couldn't find it.
https://postgr.es/m/20180629.173418.190173462
On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote:
>
> Hi,
>
> I was looking the shared memory stats patch again.
Can you point me to this thread? I looked for it but couldn't find it.
--
greg
On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada wrote:
>
> >
> > 6. src/backend/postmaster/pgstat.c - function name
> >
> > +/* --
> > + * pgstat_reset_subscription_counter() -
> > + *
> > + * Tell the statistics collector to reset a single subscription
> > + * counter, or all subscription
Below are my review comments for the v3 patch.
==
1. Commit message
(An earlier review comment [Peter-v2] #2 was only partly fixed)
"there are no longer any relation information" --> "there is no longer
any relation information"
~~~
2. doc/src/sgml/monitoring.sgml
+
pg_stat_subscri
Hi,
Thank you for the comments!
On Thu, Feb 24, 2022 at 4:20 PM tanghy.f...@fujitsu.com
wrote:
>
> On Thu, Feb 24, 2022 9:33 AM Masahiko Sawada wrote:
> >
> > Thank you for the comments! I've attached the latest version patch
> > that incorporated all comments I got so far. The primary change f
Hi,
On 2022-02-24 13:23:55 +0100, Peter Eisentraut wrote:
> On 24.02.22 12:46, Masahiko Sawada wrote:
> > > We have a view called pg_stat_activity, which is very well known. From
> > > that perspective, "activity" means what is happening right now or what
> > > has happened most recently. The re
On Thu, Feb 24, 2022 at 9:23 PM Peter Eisentraut
wrote:
>
>
> On 24.02.22 12:46, Masahiko Sawada wrote:
> >> We have a view called pg_stat_activity, which is very well known. From
> >> that perspective, "activity" means what is happening right now or what
> >> has happened most recently. The rew
On 24.02.22 12:46, Masahiko Sawada wrote:
We have a view called pg_stat_activity, which is very well known. From
that perspective, "activity" means what is happening right now or what
has happened most recently. The reworked view in this patch does not
contain that (we already have pg_stat_su
On Thu, Feb 24, 2022 at 9:05 PM Amit Kapila wrote:
>
> On Thu, Feb 24, 2022 at 2:24 PM Peter Smith wrote:
> >
> > 10. src/backend/replication/logical/worker.c
> >
> > (from my previous [Peter-v1] #9)
> >
> > >> + /* Report the worker failed during table synchronization */
> > >> + pgstat_report_s
On Thu, Feb 24, 2022 at 2:24 PM Peter Smith wrote:
>
> 9. src/backend/postmaster/pgstat.c - pgstat_recv_subscription_purge
>
> static void
> pgstat_recv_subscription_purge(PgStat_MsgSubscriptionPurge *msg, int len)
> {
> /* Return if we don't have replication subscription statistics */
> if (subsc
On Thu, Feb 24, 2022 at 6:53 PM Peter Eisentraut
wrote:
>
> On 24.02.22 02:32, Masahiko Sawada wrote:
> > On Wed, Feb 23, 2022 at 12:08 PM Peter Smith wrote:
> >>
> >> Hi. Below are my review comments for the v1 patch.
> >
> > Thank you for the comments! I've attached the latest version patch
> >
On 24.02.22 02:32, Masahiko Sawada wrote:
On Wed, Feb 23, 2022 at 12:08 PM Peter Smith wrote:
Hi. Below are my review comments for the v1 patch.
Thank you for the comments! I've attached the latest version patch
that incorporated all comments I got so far. The primary change from
the previou
On 23.02.22 03:14, Andres Freund wrote:
Why are the stats stored in the per-database stats file / as a second level
below the database? While they're also associated with a database, it's a
global catalog, so it seems to make more sense to have them "live" globally as
well?
pg_subscription bein
On 21.02.22 17:17, Andres Freund wrote:
Hi,
On 2022-02-21 14:49:01 +0530, Amit Kapila wrote:
On Mon, Feb 21, 2022 at 1:18 PM Andres Freund wrote:
* stats_reset (Time at which these statistics were last reset)
The view name could be pg_stat_subscription_lrep,
pg_stat_logical_replication, or s
Hi. Below are my review comments for the v2 patch.
==
1. Commit message
This patch changes the pg_stat_subscription_workers view (introduced
by commit 8d74fc9) so that it stores only statistics counters:
apply_error_count and sync_error_count, and has one entry for
subscription.
SUGGESTION
On Thu, Feb 24, 2022 9:33 AM Masahiko Sawada wrote:
>
> Thank you for the comments! I've attached the latest version patch
> that incorporated all comments I got so far. The primary change from
> the previous version is that the subscription statistics live globally
> rather than per-database.
>
On Thu, Feb 24, 2022 at 7:03 AM Masahiko Sawada wrote:
>
> > ~~~
> >
> > 6. doc/src/sgml/monitoring.sgml - why two counters?
> >
> > Please forgive this noob question...
> >
> > I see there are 2 error_count columns (one for each kind of worker)
> > but I was wondering why it is useful for users
Hi,
On Wed, Feb 23, 2022 at 12:08 PM Peter Smith wrote:
>
> Hi. Below are my review comments for the v1 patch.
Thank you for the comments! I've attached the latest version patch
that incorporated all comments I got so far. The primary change from
the previous version is that the subscription st
Below are my review comments just for the v1 patch test code.
==
1. "table sync error" versus "tablesync error"
+# Wait for the table sync error to be reported.
+$node_subscriber->poll_query_until(
+ 'postgres',
+ qq[
+SELECT apply_error_count = 0 AND sync_error_count > 0
+FROM pg_stat_subsc
On Tue, Feb 22, 2022 at 8:02 PM Masahiko Sawada wrote:
>
> On Tue, Feb 22, 2022 at 6:53 PM Amit Kapila wrote:
> >
>
> > 3. Can we send error stats pgstat_report_stat() as that will be called
> > via proc_exit() path. We can set the phase (apply/sync) in
> > apply_error_callback_arg and then use t
Hi. Below are my review comments for the v1 patch.
==
1. Commit message - wording
As the result of the discussion, we've concluded that the stats
collector is not an appropriate place to store the error information of
subscription workers.
SUGGESTION
It was decided (refer to the Discussion
On Tue, Feb 22, 2022 1:45 PM Masahiko Sawada wrote:
>
> I've attached a patch that changes pg_stat_subscription_workers view.
> It removes non-cumulative values such as error details such as
> error-XID and the error message from the view, and consequently the
> view now has only cumulative stati
On Wed, Feb 23, 2022 at 11:14 AM Andres Freund wrote:
>
> Hi,
>
> On 2022-02-22 14:45:19 +0900, Masahiko Sawada wrote:
> > I've attached a patch that changes pg_stat_subscription_workers view.
>
> Thanks for working on this!
>
> Why are the stats stored in the per-database stats file / as a second
Hi,
On 2022-02-22 14:45:19 +0900, Masahiko Sawada wrote:
> I've attached a patch that changes pg_stat_subscription_workers view.
Thanks for working on this!
Why are the stats stored in the per-database stats file / as a second level
below the database? While they're also associated with a databa
On Tuesday, February 22, 2022 11:47 PM Masahiko Sawada
wrote:
> On Tue, Feb 22, 2022 at 9:22 PM osumi.takami...@fujitsu.com
> wrote:
> > (4) doc/src/sgml/monitoring.sgml
> >
> >
> > role="column_definition">
> > - last_error_time timestamp with
> time zone
> > + sync_e
On Tue, Feb 22, 2022 at 9:22 PM osumi.takami...@fujitsu.com
wrote:
>
> On Tuesday, February 22, 2022 2:45 PM Masahiko Sawada
> wrote:
> > I've attached a patch that changes pg_stat_subscription_workers view.
> > It removes non-cumulative values such as error details such as error-XID and
> > the
On Tue, Feb 22, 2022 at 6:53 PM Amit Kapila wrote:
>
> On Tue, Feb 22, 2022 at 11:15 AM Masahiko Sawada
> wrote:
> >
> > I've attached a patch that changes pg_stat_subscription_workers view.
> > It removes non-cumulative values such as error details such as
> > error-XID and the error message fr
On Tuesday, February 22, 2022 2:45 PM Masahiko Sawada
wrote:
> I've attached a patch that changes pg_stat_subscription_workers view.
> It removes non-cumulative values such as error details such as error-XID and
> the error message from the view, and consequently the view now has only
> cumulativ
On Tue, Feb 22, 2022 at 11:15 AM Masahiko Sawada wrote:
>
> I've attached a patch that changes pg_stat_subscription_workers view.
> It removes non-cumulative values such as error details such as
> error-XID and the error message from the view, and consequently the
> view now has only cumulative st
On Mon, Feb 21, 2022 at 6:19 PM Amit Kapila wrote:
>
> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund wrote:
> >
> > On 2022-02-21 12:39:31 +0530, Amit Kapila wrote:
> > > Fair enough. Then, how about the following keeping the following
> > > information:
> >
> > Mostly sounds good.
> >
> >
> > >
On Mon, Feb 21, 2022 at 9:37 PM David G. Johnston
wrote:
>
> On Mon, Feb 21, 2022 at 2:19 AM Amit Kapila wrote:
>>
>> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund wrote:
>>
>> > > The view name could be pg_stat_subscription_lrep,
>> > > pg_stat_logical_replication, or something on those lines.
On Sun, Feb 20, 2022 at 10:10 PM Amit Kapila
wrote:
> On Sat, Feb 19, 2022 at 10:35 PM David G. Johnston
> wrote:
> >
> > On Sat, Feb 19, 2022 at 9:37 AM Andres Freund
> wrote:
> >>
> >> IMO the type of information you'd want for apply failures is
> substantially
> >>
> >> different enough from
Hi,
On 2022-02-21 14:49:01 +0530, Amit Kapila wrote:
> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund wrote:
> > > * stats_reset (Time at which these statistics were last reset)
> > >
> > > The view name could be pg_stat_subscription_lrep,
> > > pg_stat_logical_replication, or something on those l
On Mon, Feb 21, 2022 at 2:19 AM Amit Kapila wrote:
> On Mon, Feb 21, 2022 at 1:18 PM Andres Freund wrote:
>
> > > The view name could be pg_stat_subscription_lrep,
> > > pg_stat_logical_replication, or something on those lines.
> >
> > pg_stat_subscription_stats :)
> >
>
> Having *stat* two time
On Mon, Feb 21, 2022 at 1:18 PM Andres Freund wrote:
>
> On 2022-02-21 12:39:31 +0530, Amit Kapila wrote:
> > Fair enough. Then, how about the following keeping the following
> > information:
>
> Mostly sounds good.
>
>
> > * subid (subscription id)
> > * subname (subscription name)
>
> Coming fr
Hi,
On 2022-02-21 12:39:31 +0530, Amit Kapila wrote:
> Fair enough. Then, how about the following keeping the following information:
Mostly sounds good.
> * subid (subscription id)
> * subname (subscription name)
Coming from catalog, via join, I assume?
> * sync_error_count/sync_failure_coun
On Mon, Feb 21, 2022 at 11:04 AM Andres Freund wrote:
>
> On 2022-02-21 12:56:46 +0900, Masahiko Sawada wrote:
>
> > To take a precedent, we used to store accumulative statistics such as
> > spill_txns to pg_stat_replication, but then for the same reason we moved
> > those statistics to the new st
Hi,
On 2022-02-21 12:56:46 +0900, Masahiko Sawada wrote:
> > The patch you referenced [1] should just store the stats in the
> > pg_stat_subscription view, not pg_stat_subscription_workers.
> >
> > It *does* make sense to keep stats about the number of table syncs that
> > failed
> > etc. But tha
On Sat, Feb 19, 2022 at 10:35 PM David G. Johnston
wrote:
>
> On Sat, Feb 19, 2022 at 9:37 AM Andres Freund wrote:
>>
>> IMO the type of information you'd want for apply failures is substantially
>>
>> different enough from worker failures that I don't really see the temptation
>> to put them in
On Sat, Feb 19, 2022 at 10:07 PM Andres Freund wrote:
>
>
> I also still think that _worker shouldn't be part of any of the naming
> here.
>
Okay, the other options that comes to mind for this are:
pg_subscription_replication_error, or
pg_subscription_lreplication_error, or pg_subscription_lrep_e
On Sun, Feb 20, 2022 at 1:02 AM Andres Freund wrote:
>
> Hi,
>
> On 2022-02-19 22:38:19 +0900, Masahiko Sawada wrote:
> > On Sat, Feb 19, 2022 at 5:32 AM Andres Freund wrote:
> > > On 2022-02-18 17:26:04 +0900, Masahiko Sawada wrote:
> > > > With this change, pg_stat_subscription_workers will be
On Sat, Feb 19, 2022 at 9:37 AM Andres Freund wrote:
> IMO the type of information you'd want for apply failures is substantially
>
different enough from worker failures that I don't really see the temptation
> to put them in the same table.
>
>
It's an error message and a transaction LSN in both
Hi,
On 2022-02-19 09:16:40 -0700, David G. Johnston wrote:
> On Sat, Feb 19, 2022 at 9:02 AM Andres Freund wrote:
> > Even leaving everything else aside, a key of (dboid, subid, subrelid),
> > where
> > subrelid can be NULL, but where (dboid, subid) is *not* unique, imo is poor
> > relational des
On Sat, Feb 19, 2022 at 9:02 AM Andres Freund wrote:
>
> Even leaving everything else aside, a key of (dboid, subid, subrelid),
> where
> subrelid can be NULL, but where (dboid, subid) is *not* unique, imo is poor
> relational design. What is the justification for mixing relation specific
> and
Hi,
On 2022-02-19 22:38:19 +0900, Masahiko Sawada wrote:
> On Sat, Feb 19, 2022 at 5:32 AM Andres Freund wrote:
> > On 2022-02-18 17:26:04 +0900, Masahiko Sawada wrote:
> > > With this change, pg_stat_subscription_workers will be like:
> > >
> > > * subid
> > > * subname
> > > * subrelid
> > > *
On Sat, Feb 19, 2022 at 4:47 AM David G. Johnston
wrote:
>
> On Fri, Feb 18, 2022 at 1:26 AM Masahiko Sawada wrote:
>>
>>
>> Here is the summary of the discussion, changes, and plan.
>>
>> 1. Move some error information such as the error message to a new
>> system catalog, pg_subscription_error.
On Sat, Feb 19, 2022 at 10:37 PM Amit Kapila wrote:
>
> On Sat, Feb 19, 2022 at 6:51 PM David G. Johnston
> wrote:
> >
> > On Saturday, February 19, 2022, Amit Kapila wrote:
> >>
> >> On Sat, Feb 19, 2022 at 1:17 AM David G. Johnston
> >> wrote:
> >> >
> >> > On Fri, Feb 18, 2022 at 1:26 AM Mas
On Sat, Feb 19, 2022 at 1:17 AM David G. Johnston
wrote:
>
> On Fri, Feb 18, 2022 at 1:26 AM Masahiko Sawada wrote:
>>
>> 5. Record skipped data to the system catalog, say
>> pg_subscription_conflict_history so that there is a chance to analyze
>> and recover.
>
>
>> We can discuss the
>> details
On Sat, Feb 19, 2022 at 5:32 AM Andres Freund wrote:
>
> Hi,
>
> On 2022-02-18 17:26:04 +0900, Masahiko Sawada wrote:
> > With this change, pg_stat_subscription_workers will be like:
> >
> > * subid
> > * subname
> > * subrelid
> > * error_count
> > * last_error_timestamp
>
> > This view will be e
On Sat, Feb 19, 2022 at 6:51 PM David G. Johnston
wrote:
>
> On Saturday, February 19, 2022, Amit Kapila wrote:
>>
>> On Sat, Feb 19, 2022 at 1:17 AM David G. Johnston
>> wrote:
>> >
>> > On Fri, Feb 18, 2022 at 1:26 AM Masahiko Sawada
>> > wrote:
>> >>
>> >>
>> >> Here is the summary of the d
On Saturday, February 19, 2022, Amit Kapila wrote:
> On Sat, Feb 19, 2022 at 1:17 AM David G. Johnston
> wrote:
> >
> > On Fri, Feb 18, 2022 at 1:26 AM Masahiko Sawada
> wrote:
> >>
> >>
> >> Here is the summary of the discussion, changes, and plan.
> >>
> >> 1. Move some error information such
On Sat, Feb 19, 2022 at 1:17 AM David G. Johnston
wrote:
>
> On Fri, Feb 18, 2022 at 1:26 AM Masahiko Sawada wrote:
>>
>>
>> Here is the summary of the discussion, changes, and plan.
>>
>> 1. Move some error information such as the error message to a new
>> system catalog, pg_subscription_error.
Hi,
On 2022-02-18 17:26:04 +0900, Masahiko Sawada wrote:
> With this change, pg_stat_subscription_workers will be like:
>
> * subid
> * subname
> * subrelid
> * error_count
> * last_error_timestamp
> This view will be extended by adding transaction statistics proposed
> on another thread[1].
I
On Fri, Feb 18, 2022 at 1:26 AM Masahiko Sawada
wrote:
>
> Here is the summary of the discussion, changes, and plan.
>
> 1. Move some error information such as the error message to a new
> system catalog, pg_subscription_error. The pg_subscription_error table
> would have the following columns:
>
On Wed, Feb 16, 2022 at 8:36 PM Masahiko Sawada wrote:
>
> On Wed, Feb 16, 2022 at 5:49 PM Amit Kapila wrote:
> >
> > On Tue, Feb 15, 2022 at 11:56 PM Andres Freund wrote:
> > >
> > > On 2022-02-03 13:33:08 +0900, Masahiko Sawada wrote:
> > > > I see that important information such as error-XID
On Wed, Feb 16, 2022 at 5:49 PM Amit Kapila wrote:
>
> On Tue, Feb 15, 2022 at 11:56 PM Andres Freund wrote:
> >
> > On 2022-02-03 13:33:08 +0900, Masahiko Sawada wrote:
> > > I see that important information such as error-XID that can be used
> > > for ALTER SUBSCRIPTION SKIP needs to be stored
On Wed, Feb 16, 2022 at 12:01 AM Robert Haas wrote:
>
> On Thu, Jan 27, 2022 at 12:46 AM Andres Freund wrote:
> > Only if either the user wants to drop all stats, or somehow knows the oids
> > of
> > already dropped tables...
>
> If it's really true that we can end up storing data for dropped
>
On Tue, Feb 15, 2022 at 11:56 PM Andres Freund wrote:
>
> On 2022-02-03 13:33:08 +0900, Masahiko Sawada wrote:
> > I see that important information such as error-XID that can be used
> > for ALTER SUBSCRIPTION SKIP needs to be stored in a reliable way, and
> > using system catalogs is a reasonable
On Tue, Feb 15, 2022 at 11:47 PM Andres Freund wrote:
>
> Hi,
>
> On 2022-02-04 09:23:06 +0530, Amit Kapila wrote:
> > On Thu, Feb 3, 2022 at 3:25 PM Peter Eisentraut
> > wrote:
> > >
> > > On 02.02.22 07:54, Amit Kapila wrote:
> > >
> > > > Sure, but is this the reason you want to store all the
On Thu, Jan 27, 2022 at 12:46 AM Andres Freund wrote:
> Only if either the user wants to drop all stats, or somehow knows the oids of
> already dropped tables...
If it's really true that we can end up storing data for dropped
objects, I think that's not acceptable and needs to be fixed.
I don't
Hi,
On 2022-02-03 13:33:08 +0900, Masahiko Sawada wrote:
> I see that important information such as error-XID that can be used
> for ALTER SUBSCRIPTION SKIP needs to be stored in a reliable way, and
> using system catalogs is a reasonable way for this purpose. But it's
> still unclear to me why al
Hi,
On 2022-02-04 09:23:06 +0530, Amit Kapila wrote:
> On Thu, Feb 3, 2022 at 3:25 PM Peter Eisentraut
> wrote:
> >
> > On 02.02.22 07:54, Amit Kapila wrote:
> >
> > > Sure, but is this the reason you want to store all the error info in
> > > the system catalog? I agree that providing more error
Hi,
On 2022-02-03 14:35:10 +0900, Masahiko Sawada wrote:
> Yes, but if we use shmem IPC, we need to allocate shared memory for
> them based on the number of subscriptions, not logical replication
> workers (i.e., max_logical_replication_workers). So we cannot estimate
> memory in the beginning. Al
On Thu, Feb 3, 2022 at 2:35 PM Masahiko Sawada wrote:
>
> On Thu, Feb 3, 2022 at 1:48 PM David G. Johnston
> wrote:
> >
> > On Wednesday, February 2, 2022, Masahiko Sawada
> > wrote:
> >>
> >> and have other error
> >> information in pg_stat_subscription_workers view.
> >
> >
> > What benefit i
On Thu, Feb 3, 2022 at 3:25 PM Peter Eisentraut
wrote:
>
> On 02.02.22 07:54, Amit Kapila wrote:
>
> > Sure, but is this the reason you want to store all the error info in
> > the system catalog? I agree that providing more error info could be
> > useful and also possibly the previously failed (ap
On 02.02.22 07:54, Amit Kapila wrote:
Where do you propose to store this information?
pg_subscription_worker
The error message and context is very important. Just make sure it is only non-null when
the worker state is "syncing failed" (or whatever term we use).
We could name the table s
On Thu, Feb 3, 2022 at 1:48 PM David G. Johnston
wrote:
>
> On Wednesday, February 2, 2022, Masahiko Sawada wrote:
>>
>> and have other error
>> information in pg_stat_subscription_workers view.
>
>
> What benefit is there to keeping the existing collector-based
> pg_stat_subscripiton_workers vi
On Wednesday, February 2, 2022, Masahiko Sawada
wrote:
> and have other error
> information in pg_stat_subscription_workers view.
>
What benefit is there to keeping the existing collector-based
pg_stat_subscripiton_workers view? If we re-write it using shmem IPC then
we might as well put everyt
On Wed, Feb 2, 2022 at 4:36 PM David G. Johnston
wrote:
>
> On Tue, Feb 1, 2022 at 11:55 PM Amit Kapila wrote:
>>
>> On Wed, Feb 2, 2022 at 9:41 AM David G. Johnston
>> wrote:
>> >
>> > On Tue, Feb 1, 2022 at 8:07 PM Amit Kapila wrote:
>> >>
>> >> On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada
On Wed, Feb 2, 2022 at 5:08 AM Amit Kapila wrote:
> On Wed, Feb 2, 2022 at 1:06 PM David G. Johnston
> wrote:
>
> ...
> >
> > I already explained that the concept of err_cnt is not useful. The fact
> that you include it here makes me think you are still thinking that this
> all somehow is meant
On Wed, Feb 2, 2022 at 1:06 PM David G. Johnston
wrote:
>
> On Tue, Feb 1, 2022 at 11:55 PM Amit Kapila wrote:
>>
>> On Wed, Feb 2, 2022 at 9:41 AM David G. Johnston
>> wrote:
>> >
>> > On Tue, Feb 1, 2022 at 8:07 PM Amit Kapila wrote:
>> >>
>> >> On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada
On Tue, Feb 1, 2022 at 11:55 PM Amit Kapila wrote:
> On Wed, Feb 2, 2022 at 9:41 AM David G. Johnston
> wrote:
> >
> > On Tue, Feb 1, 2022 at 8:07 PM Amit Kapila
> wrote:
> >>
> >> On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada
> wrote:
> >>
> >> >
> >> > I see that it's better to use a bette
On Wed, Feb 2, 2022 at 9:41 AM David G. Johnston
wrote:
>
> On Tue, Feb 1, 2022 at 8:07 PM Amit Kapila wrote:
>>
>> On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada
>> wrote:
>>
>> >
>> > I see that it's better to use a better IPC for ALTER SUBSCRIPTION SKIP
>> > feature to pass error-XID or err
On Tue, Feb 1, 2022 at 8:07 PM Amit Kapila wrote:
> On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada
> wrote:
>
> >
> > I see that it's better to use a better IPC for ALTER SUBSCRIPTION SKIP
> > feature to pass error-XID or error-LSN information to the worker
> > whereas I'm also not sure of the
On Tue, Feb 1, 2022 at 11:47 AM Masahiko Sawada wrote:
>
> On Fri, Jan 28, 2022 at 2:59 PM Amit Kapila wrote:
> >
> > On Fri, Jan 28, 2022 at 1:49 AM David G. Johnston
> > wrote:
> > >
> > >
> > > In short, it was convenient to use the statistics collector here even if
> > > doing so resulted i
On Fri, Jan 28, 2022 at 2:59 PM Amit Kapila wrote:
>
> On Fri, Jan 28, 2022 at 1:49 AM David G. Johnston
> wrote:
> >
> > On Thu, Jan 27, 2022 at 5:08 AM Amit Kapila wrote:
> >>
> >> On Thu, Jan 27, 2022 at 11:16 AM Andres Freund wrote:
> >> >
> >> > On 2022-01-25 20:27:07 +0900, Masahiko Sawad
On Fri, Jan 28, 2022 at 1:49 AM David G. Johnston
wrote:
>
> On Thu, Jan 27, 2022 at 5:08 AM Amit Kapila wrote:
>>
>> On Thu, Jan 27, 2022 at 11:16 AM Andres Freund wrote:
>> >
>> > On 2022-01-25 20:27:07 +0900, Masahiko Sawada wrote:
>> >
>> > > There will be some challenges in a case where upd
On Thu, Jan 27, 2022 at 2:15 PM Andres Freund wrote:
> Another related thing is that using a 32bit xid for allowing skipping is a
> bad
> idea anyway - we shouldn't adding new interfaces with xid wraparound
> dangers -
> it's getting more and more common to have multiple wraparounds a day. An
>
1 - 100 of 106 matches
Mail list logo
