On 2021-Aug-17, Justin Pryzby wrote:
> I suggest the attached (which partially reverts the revert), to allow showing
> correct data for analyze_count and last_analyzed.
Yeah, that makes sense and my keeping of the pg_stat_all_tables entries
seems pretty useless without this change. I have pushed
On Fri, Aug 20, 2021 at 07:55:13AM -0500, Justin Pryzby wrote:
> On Tue, Aug 17, 2021 at 06:30:18AM -0500, Justin Pryzby wrote:
> > On Mon, Aug 16, 2021 at 05:28:10PM -0500, Justin Pryzby wrote:
> > > On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote:
> > > > On 2021-Aug-16, Álvaro Her
On Tue, Aug 17, 2021 at 06:30:18AM -0500, Justin Pryzby wrote:
> On Mon, Aug 16, 2021 at 05:28:10PM -0500, Justin Pryzby wrote:
> > On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote:
> > > On 2021-Aug-16, Álvaro Herrera wrote:
> > >
> > > > Here's the reversal patch for the 14 branch.
On Mon, Aug 16, 2021 at 05:28:10PM -0500, Justin Pryzby wrote:
> On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote:
> > On 2021-Aug-16, Álvaro Herrera wrote:
> >
> > > Here's the reversal patch for the 14 branch. (It applies cleanly to
> > > master, but the unused member of PgStat_St
Hi,
On 2021-08-16 13:13:55 -0400, Álvaro Herrera wrote:
> Another possible problem is that before the revert, we accept
> ALTER TABLE some_partitioned_table SET (autovacuum_enabled=on/off);
> (also autovacuum_analyze_scale_factor and autovacuum_analyze_threshold)
> but after the revert this is wil
Hi,
On 2021-08-16 17:42:48 -0400, Álvaro Herrera wrote:
> On 2021-Aug-16, Álvaro Herrera wrote:
>
> > Here's the reversal patch for the 14 branch. (It applies cleanly to
> > master, but the unused member of PgStat_StatTabEntry needs to be
> > removed and catversion bumped).
>
> I have pushed this
On Mon, Aug 16, 2021 at 05:42:48PM -0400, Álvaro Herrera wrote:
> On 2021-Aug-16, Álvaro Herrera wrote:
>
> > Here's the reversal patch for the 14 branch. (It applies cleanly to
> > master, but the unused member of PgStat_StatTabEntry needs to be
> > removed and catversion bumped).
>
> I have pu
On 2021-Aug-16, Álvaro Herrera wrote:
> Here's the reversal patch for the 14 branch. (It applies cleanly to
> master, but the unused member of PgStat_StatTabEntry needs to be
> removed and catversion bumped).
I have pushed this to both branches. (I did not remove the item from
the release notes
Another possible problem is that before the revert, we accept
ALTER TABLE some_partitioned_table SET (autovacuum_enabled=on/off);
(also autovacuum_analyze_scale_factor and autovacuum_analyze_threshold)
but after the revert this is will throw a syntax error. What do people
think we should do about
On 2021-Aug-16, Tom Lane wrote:
> =?utf-8?Q?=C3=81lvaro?= Herrera writes:
> > Here's the reversal patch for the 14 branch. (It applies cleanly to
> > master, but the unused member of PgStat_StatTabEntry needs to be
> > removed and catversion bumped).
>
> I don't follow the connection to catvers
=?utf-8?Q?=C3=81lvaro?= Herrera writes:
> Here's the reversal patch for the 14 branch. (It applies cleanly to
> master, but the unused member of PgStat_StatTabEntry needs to be
> removed and catversion bumped).
I don't follow the connection to catversion?
I agree that we probably don't want to
Here's the reversal patch for the 14 branch. (It applies cleanly to
master, but the unused member of PgStat_StatTabEntry needs to be
removed and catversion bumped).
--
Álvaro Herrera 39°49'30"S 73°17'W — https://www.EnterpriseDB.com/
Maybe there's lots of data loss but the records of
On 2021-Aug-13, Álvaro Herrera wrote:
> Some doc changes are pending, and some more commentary in parts of the
> code, but I think this is much more sensible. I do lament the lack of
> a syscache for pg_inherits.
Thinking about this again, this one here is the killer problem, I think;
this behav
Here is a proposal for 14. This patch has four main changes:
* The mod counts are only propagated to the topmost parent, not to each
ancestor. This means that we'll only analyze the topmost partitioned table and
not each intermediate partitioned table; seems a good compromise to avoid
samplin
Hi,
On 2021-08-11 18:33:07 -0400, Alvaro Herrera wrote:
> After thinking about the described issues for a while, my proposal is to
> completely revamp the way this feature works. See below.
>
> Now, the proposal seems awfully invasive, but it's *the* way I see to
> avoid the pgstat traffic. For
After thinking about the described issues for a while, my proposal is to
completely revamp the way this feature works. See below.
Now, the proposal seems awfully invasive, but it's *the* way I see to
avoid the pgstat traffic. For pg14, maybe we can live with it, and just
use the smaller patches
On 2021-Aug-09, Alvaro Herrera wrote:
> > 3) What is the goal of the autovac_refresh_stats() after the loop doing
> >pgstat_report_anl_ancestors()? I think it'll be common that the stats
> >collector hasn't even processed the incoming messages by that point, not
> > to
> >speak of act
On 2021-Aug-10, Alvaro Herrera wrote:
> I bring a radical proposal that may be sufficient to close this
> particular hole. What if we made partition only affected their
> top-level parents to become auto-analyzed, and not any intermediate
> ancestors? Any intermediate partitioned partitions coul
On 2021-Aug-09, Andres Freund wrote:
> I don't agree. There's a difference between this happening after a manual
> ANALYZE on partition roots, and this continuously happening in production
> workloads due to auto-analyzes...
Hmm. That's not completely untrue.
I bring a radical proposal that may
Hi,
On 2021-08-09 16:02:33 -0400, Alvaro Herrera wrote:
> On 2021-Jul-27, Andres Freund wrote:
>
> > Isn't this going to create a *lot* of redundant sampling? Especially if you
> > have any sort of nested partition tree. In the most absurd case a partition
> > with n parents will get sampled n t
Hello,
On 2021-Jul-22, Andres Freund wrote:
> 1) Somehow it seems like a violation to do stuff like
>get_partition_ancestors() in pgstat.c. It's nothing I can't live with, but
>it feels a bit off. Would likely not be too hard to address, e.g. by just
>putting some of pgstat_report_anl
Hi
On 2021-Jul-27, Andres Freund wrote:
> Isn't this going to create a *lot* of redundant sampling? Especially if you
> have any sort of nested partition tree. In the most absurd case a partition
> with n parents will get sampled n times, solely due to changes to itself.
It seems to me that you
At Thu, 29 Jul 2021 18:03:55 -0700, Andres Freund wrote in
> And if one instead inverts the order of pgstat_report_analyze() and
> pgstat_report_anl_ancestors() one gets a slightly different problem: A manual
> ANALYZE of the partition root results in the partition root having a non-zero
> change
On 7/29/21 9:03 PM, Andres Freund wrote:
> Hi,
>
> CCing RMT because I think we need to do something about this for v14.
Thanks. We are now aware of it.
[...]
> I don't think the code as is is fit for v14. It looks like it was rewritten
> with a new approach just before the freeze ([1]), an
Hi,
CCing RMT because I think we need to do something about this for v14.
On 2021-07-27 19:23:42 -0700, Andres Freund wrote:
> On 2021-07-22 13:54:58 -0700, Andres Freund wrote:
> > On 2021-04-08 01:20:14 -0400, Alvaro Herrera wrote:
> > > On 2021-Apr-07, Alvaro Herrera wrote:
> > >
> > > > OK, I
Hi,
On 2021-07-22 13:54:58 -0700, Andres Freund wrote:
> On 2021-04-08 01:20:14 -0400, Alvaro Herrera wrote:
> > On 2021-Apr-07, Alvaro Herrera wrote:
> >
> > > OK, I bit the bullet and re-did the logic in the way I had proposed
> > > earlier in the thread: do the propagation on the collector's si
Hi,
On 2021-04-08 01:20:14 -0400, Alvaro Herrera wrote:
> On 2021-Apr-07, Alvaro Herrera wrote:
>
> > OK, I bit the bullet and re-did the logic in the way I had proposed
> > earlier in the thread: do the propagation on the collector's side, by
> > sending only the list of ancestors: the collector
Justin Pryzby writes:
> On Thu, Apr 08, 2021 at 05:56:25PM -0400, Alvaro Herrera wrote:
>> This new bit reads weird:
>>
>> +Most parameters are not supported on partitioned tables, with exceptions
>> +noted below; you may specify them for individual leaf partitions.
> "Except where noted
On Thu, Apr 08, 2021 at 05:56:25PM -0400, Alvaro Herrera wrote:
> On 2021-Apr-08, Justin Pryzby wrote:
>
> > commit 0827e8af70f4653ba17ed773f123a60eadd9f9c9
> > |This also introduces necessary reloptions support for partitioned
> > tables
> > |(autovacuum_enabled, autovacuum_analyze_scale
On 2021-Apr-08, Justin Pryzby wrote:
> commit 0827e8af70f4653ba17ed773f123a60eadd9f9c9
> |This also introduces necessary reloptions support for partitioned tables
> |(autovacuum_enabled, autovacuum_analyze_scale_factor,
> |autovacuum_analyze_threshold). It's unclear how best to docume
On Thu, Apr 08, 2021 at 01:20:14AM -0400, Alvaro Herrera wrote:
> On 2021-Apr-07, Alvaro Herrera wrote:
>
> > OK, I bit the bullet and re-did the logic in the way I had proposed
> > earlier in the thread: do the propagation on the collector's side, by
> > sending only the list of ancestors: the co
On 2021-Apr-08, Tomas Vondra wrote:
> On 4/8/21 5:27 PM, Alvaro Herrera wrote:
>
> > Same as for any other relation: ANALYZE would set it, after it's done
> > scanning the table. We would to make sure that nothing resets it to
> > empty, though, and that it doesn't cause issues elsewhere. (The p
On 4/8/21 5:27 PM, Alvaro Herrera wrote:
> On 2021-Apr-08, Tomas Vondra wrote:
>
>> On 4/8/21 5:22 AM, Alvaro Herrera wrote:
>
>>> However, I just noticed there is a huge problem, which is that the new
>>> code in relation_needs_vacanalyze() is doing find_all_inheritors(), and
>>> we don't necess
On 2021-Apr-08, Tomas Vondra wrote:
> On 4/8/21 5:22 AM, Alvaro Herrera wrote:
> > However, I just noticed there is a huge problem, which is that the new
> > code in relation_needs_vacanalyze() is doing find_all_inheritors(), and
> > we don't necessarily have a snapshot that lets us do that. Whi
On 4/8/21 5:22 AM, Alvaro Herrera wrote:
> OK, I bit the bullet and re-did the logic in the way I had proposed
> earlier in the thread: do the propagation on the collector's side, by
> sending only the list of ancestors: the collector can read the tuple
> change count by itself, to add it to eac
On 2021-Apr-07, Alvaro Herrera wrote:
> However, I just noticed there is a huge problem, which is that the new
> code in relation_needs_vacanalyze() is doing find_all_inheritors(), and
> we don't necessarily have a snapshot that lets us do that. While adding
> a snapshot acquisition at that spot
On 2021-Apr-07, Alvaro Herrera wrote:
> OK, I bit the bullet and re-did the logic in the way I had proposed
> earlier in the thread: do the propagation on the collector's side, by
> sending only the list of ancestors: the collector can read the tuple
> change count by itself, to add it to each anc
OK, I bit the bullet and re-did the logic in the way I had proposed
earlier in the thread: do the propagation on the collector's side, by
sending only the list of ancestors: the collector can read the tuple
change count by itself, to add it to each ancestor. This seems less
wasteful. Attached is
Hi,
I fixed the patch according to the following comments.
Attach the latest patch. It is based on v14 patch Alvaro attached before.
On Mon, Apr 5, 2021 at 4:08 AM Tomas Vondra
wrote:
>
> On 4/3/21 9:42 PM, Alvaro Herrera wrote:
> > Thanks for the quick rework. I like this design much better a
On 2021-Apr-07, yuzuko wrote:
> I'm working on fixing the patch according to the comments.
> I'll send it as soon as I can.
Thanks, I've been giving it a look too.
> I've been thinking about traditional inheritance, I realized that we
> need additional
> handling to support them because unlike d
Hello,
Thank you for reviewing.
I'm working on fixing the patch according to the comments.
I'll send it as soon as I can.
> On 2021-04-06 16:56:49 -0400, Alvaro Herrera wrote:
> > I think there is a good reason to treat them the same: pgstat does not
> > have a provision to keep stats both of the
Hi,
On 2021-04-06 16:56:49 -0400, Alvaro Herrera wrote:
> I think there is a good reason to treat them the same: pgstat does not
> have a provision to keep stats both of the table with children, and the
> table without children. It can only have one of those. For
> partitioning that doesn't matt
On 2021-Apr-04, Tomas Vondra wrote:
> 1) I still don't understand why inheritance and declarative partitioning
> are treated differently. Seems unnecessary nad surprising, but maybe
> there's a good reason?
I think there is a good reason to treat them the same: pgstat does not
have a provision to
On 4/4/21 9:08 PM, Tomas Vondra wrote:
> On 4/3/21 9:42 PM, Alvaro Herrera wrote:
>> Thanks for the quick rework. I like this design much better and I think
>> this is pretty close to committable. Here's a rebased copy with some
>> small cleanups (most notably, avoid calling pgstat_propagate_c
On 2021-Apr-04, Tomas Vondra wrote:
> In fact, one of the first posts in this threads links to this:
>
> https://www.postgresql.org/message-id/4823.1262132964%40sss.pgh.pa.us
>
> i.e. Tom actually proposed doing something like this back in 2009, so
> presumably he though it's desirable back then
On 4/4/21 10:05 PM, Alvaro Herrera wrote:
> On 2021-Apr-04, Tomas Vondra wrote:
>
>> 1) I still don't understand why inheritance and declarative partitioning
>> are treated differently. Seems unnecessary nad surprising, but maybe
>> there's a good reason?
>
> I suppose the rationale is that fo
On 2021-Apr-04, Tomas Vondra wrote:
> 1) I still don't understand why inheritance and declarative partitioning
> are treated differently. Seems unnecessary nad surprising, but maybe
> there's a good reason?
I suppose the rationale is that for inheritance we have always done it
that way -- similar
On 4/3/21 9:42 PM, Alvaro Herrera wrote:
> Thanks for the quick rework. I like this design much better and I think
> this is pretty close to committable. Here's a rebased copy with some
> small cleanups (most notably, avoid calling pgstat_propagate_changes
> when the partition doesn't have a tabs
Thanks for the quick rework. I like this design much better and I think
this is pretty close to committable. Here's a rebased copy with some
small cleanups (most notably, avoid calling pgstat_propagate_changes
when the partition doesn't have a tabstat entry; also, free the lists
that are allocate
Hi Tomas,
Thank you for reviewing the patch.
> Firstly, the patch propagates the changes_since_analyze values from
> do_analyze_rel, i.e. from the worker after it analyzes the relation.
> That may easily lead to cases with unnecessary analyzes - consider a
> partitioned with 4 child relations:
>
On 3/30/21 4:09 AM, Tomas Vondra wrote:
> Hi,
>
> ...
>
> We may need to "sync" the counts for individual relations in a couple
> places (e.g. after the worker is done with the leaf, it should propagate
> the remaining delta before resetting the values to 0). Maybe multi-level
> partitioning n
Hi,
I took a look at this patch. It does not apply because of 5f8727f5a67,
so a rebase is needed. But I want to talk about the general approach in
general, so it does not matter.
The thread is fairly long, both in terms of number of messages and time
(started in 2019), so let me restate my unders
On 12/14/20 8:46 PM, yuzuko wrote:
On Thu, Dec 3, 2020 at 10:28 PM Alvaro Herrera wrote:
Attach the new patch based on his patch. What do you think?
Álvaro, Justin, Kyotaro, thoughts on this latest patch?
Regards,
--
-David
da...@pgmasters.net
Hello Alvaro,
On Thu, Dec 3, 2020 at 10:28 PM Alvaro Herrera wrote:
>
> Hello Yuzuko,
>
> On 2020-Dec-02, yuzuko wrote:
>
> > The problem Horiguchi-san mentioned is as follows:
> > [explanation]
>
> Hmm, I see. So the problem is that if some ancestor is analyzed first,
> then analyze of one of i
Hello Yuzuko,
On 2020-Dec-02, yuzuko wrote:
> The problem Horiguchi-san mentioned is as follows:
> [explanation]
Hmm, I see. So the problem is that if some ancestor is analyzed first,
then analyze of one of its partition will cause a redundant analyze of
the ancestor, because the number of tupl
Hello Alvaro,
Thank you for your comments.
>
> > In second thought about the reason for the "toprel_oid". It is perhaps
> > to avoid "wrongly" propagated values to ancestors after a manual
> > ANALYZE on a partitioned table. But the same happens after an
> > autoanalyze iteration if some of the
On 2020-Nov-10, Kyotaro Horiguchi wrote:
> In second thought about the reason for the "toprel_oid". It is perhaps
> to avoid "wrongly" propagated values to ancestors after a manual
> ANALYZE on a partitioned table. But the same happens after an
> autoanalyze iteration if some of the ancestors of
I looked at both Yuzuko Hosoya's patch and Kyotaro Horiguchi's, and
think we're doing things in a quite complicated manner, which perhaps
could be done more easily.
Hosoya's patch has pgstat_report_analyze call pgstat_get_tab_entry() for
the table being vacuumed, then obtains the list of ancestors
At Thu, 5 Nov 2020 16:03:12 +0900, yuzuko wrote in
> Hi Justin,
>
> Thank you for your comments.
> I attached the latest patch(v11) to the previous email.
>
> >
> > +* Get its all ancestors to propagate changes_since_analyze
> > count.
> > +* However, when ANALY
Hi Justin,
Thank you for your comments.
I attached the latest patch(v11) to the previous email.
>
> +* Get its all ancestors to propagate changes_since_analyze
> count.
> +* However, when ANALYZE inheritance tree, we get ancestors of
> +* toprel_oi
Horiguchi-san,
Thank you for your comments.
On Fri, Oct 23, 2020 at 8:23 PM Kyotaro Horiguchi
wrote:
>
> Thanks you for the new version.
>
> At Fri, 23 Oct 2020 15:12:51 +0900, yuzuko wrote in
> > Hello,
> >
> > I reconsidered a way based on the v5 patch in line with
> > Horiguchi-san's commen
On Fri, Oct 23, 2020 at 03:12:51PM +0900, yuzuko wrote:
> Hello,
>
> I reconsidered a way based on the v5 patch in line with
> Horiguchi-san's comment.
>
> This approach is as follows:
> - A partitioned table is checked whether it needs analyze like a plain
> table in relation_needs_vacanalyze
Thanks you for the new version.
At Fri, 23 Oct 2020 15:12:51 +0900, yuzuko wrote in
> Hello,
>
> I reconsidered a way based on the v5 patch in line with
> Horiguchi-san's comment.
>
> This approach is as follows:
> - A partitioned table is checked whether it needs analyze like a plain
> tab
Hello,
I reconsidered a way based on the v5 patch in line with
Horiguchi-san's comment.
This approach is as follows:
- A partitioned table is checked whether it needs analyze like a plain
table in relation_needs_vacanalyze(). To do this, we should store
partitioned table's stats (changes_si
Horiguchi-san,
Thank you for reviewing.
On Tue, Sep 15, 2020 at 7:01 PM Kyotaro Horiguchi
wrote:
>
> At Tue, 25 Aug 2020 14:28:20 +0200, Daniel Gustafsson wrote
> in
> > > I attach the latest patch that solves the above Werror.
> > > Could you please check it again?
> >
> > This version now
At Tue, 25 Aug 2020 14:28:20 +0200, Daniel Gustafsson wrote
in
> > I attach the latest patch that solves the above Werror.
> > Could you please check it again?
>
> This version now pass the tests in the Travis pipeline as can be seen in the
> link below, and is ready to be reviewed in the upcom
> On 17 Aug 2020, at 08:11, yuzuko wrote:
>
> I'm sorry for the late reply.
>
>> This version seems to fail under Werror which is used in the Travis builds:
>>
>> autovacuum.c: In function ‘relation_needs_vacanalyze’:
>> autovacuum.c:3117:59: error: ‘reltuples’ may be used uninitialized in this
I'm sorry for the late reply.
> This version seems to fail under Werror which is used in the Travis builds:
>
> autovacuum.c: In function ‘relation_needs_vacanalyze’:
> autovacuum.c:3117:59: error: ‘reltuples’ may be used uninitialized in this
> function [-Werror=maybe-uninitialized]
>anlthre
> On 6 Jul 2020, at 12:35, yuzuko wrote:
>
>> On Wed, Jul 1, 2020 at 6:26 PM Daniel Gustafsson wrote:
>>
>>> On 21 Apr 2020, at 18:21, yuzuko wrote:
>>
>>> I'll update the patch soon.
>>
>> Do you have an updated version to submit? The previous patch no longer
>> applies
>> to HEAD, so I'm
> On Wed, Jul 1, 2020 at 6:26 PM Daniel Gustafsson wrote:
>
> > On 21 Apr 2020, at 18:21, yuzuko wrote:
>
> > I'll update the patch soon.
>
> Do you have an updated version to submit? The previous patch no longer
> applies
> to HEAD, so I'm marking this entry Waiting on Author in the meantime.
> On 21 Apr 2020, at 18:21, yuzuko wrote:
> I'll update the patch soon.
Do you have an updated version to submit? The previous patch no longer applies
to HEAD, so I'm marking this entry Waiting on Author in the meantime.
cheers ./daniel
On Sat, Apr 25, 2020 at 11:13 PM Justin Pryzby wrote:
>
> On Wed, Mar 18, 2020 at 11:30:39AM -0500, Justin Pryzby wrote:
> > In the past, I think there's was talk that maybe someone would invent a
> > clever
> > way to dynamically combine all the partitions' statistics, so analyzing the
> > paren
On Wed, Mar 18, 2020 at 11:30:39AM -0500, Justin Pryzby wrote:
> In the past, I think there's was talk that maybe someone would invent a clever
> way to dynamically combine all the partitions' statistics, so analyzing the
> parent wasn't needed. [...]
I happened across the thread I was referring t
Hello,
On Sat, Apr 18, 2020 at 2:08 PM Justin Pryzby wrote:
>
> On Fri, Apr 17, 2020 at 10:09:07PM +0900, Amit Langote wrote:
> > On Thu, Apr 16, 2020 at 11:19 PM Justin Pryzby wrote:
> > > On Thu, Apr 16, 2020 at 06:16:45PM +0900, yuzuko wrote:
> > > I don't think that adequately allows what's
On Fri, Apr 17, 2020 at 10:09:07PM +0900, Amit Langote wrote:
> On Thu, Apr 16, 2020 at 11:19 PM Justin Pryzby wrote:
> > On Thu, Apr 16, 2020 at 06:16:45PM +0900, yuzuko wrote:
> > I don't think that adequately allows what's needed.
...(paragraph with my typos elided)...
> > For example, say a ne
On Thu, Apr 16, 2020 at 11:19 PM Justin Pryzby wrote:
> On Thu, Apr 16, 2020 at 06:16:45PM +0900, yuzuko wrote:
> > The latest patch lets users set different autovacuum configuration for
> > each partitioned
> > tables like this,
> > create table p3(i int) partition by range(i) with
> >(auto
On Thu, Apr 16, 2020 at 06:16:45PM +0900, yuzuko wrote:
> > I think it ought to be possible to configure this feature such that an
> > auto-analyze on any child partition would trigger analyze of the parent. I
> > think that would be important for maintaining accurate stats of the
> > partition
>
Hi Justin,
Thank you for commens.
On Tue, Apr 7, 2020 at 12:32 PM Justin Pryzby wrote:
>
> Not sure if you saw my earlier message ?
>
I'm sorry, I didn't notice for a while.
> I think it ought to be possible to configure this feature such that an
> auto-analyze on any child partition would trig
Not sure if you saw my earlier message ?
I think it ought to be possible to configure this feature such that an
auto-analyze on any child partition would trigger analyze of the parent. I
think that would be important for maintaining accurate stats of the partition
key column for many cases involv
Hi Alvaro,
Thank you for your comments.
> I'm confused about some error messages in the regression test when a
> column is mentioned twice, that changed from mentioning the table named
> in the vacuum command, to mentioning the first partition. Is that
> because you changed an lappend() to lcons(
Regarding this patch:
+* the ANALYZE message as it resets the partition's changes_since_analze
=> analyze
+* If the relation is a partitioned table, we must add up children's
childrens'
The approach in general:
I see an issue for timeseries data, where only the most recent parti
On 2020-Mar-18, yuzuko wrote:
> > I think if we analyze partition tree in order from leaf partitions
> > to root table, this problem can be fixed.
> > What do you think about it?
>
> Attach the new patch fixes the above problem.
Thanks for the new version.
I'm confused about some error messages
Hello,
> > > > + */
> > > > + if (IsAutoVacuumWorkerProcess() &&
> > > > + rel->rd_rel->relispartition &&
> > > > + !(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE))
> > >
> > > I'm not sure I understand why we do this only on autovac. Why not all
> > > analyze
Hello,
Thank you for reviewing.
> > > + */
> > > + if (IsAutoVacuumWorkerProcess() &&
> > > + rel->rd_rel->relispartition &&
> > > + !(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE))
> >
> > I'm not sure I understand why we do this only on autovac. Why not all
On Fri, Feb 28, 2020 at 11:25 AM Alvaro Herrera
wrote:
> > /*
> > + * If the relation is a partitioned table, we must add up children's
> > + * reltuples.
> > + */
> > + if (classForm->relkind == RELKIND_PARTITIONED_TABLE)
> > + {
> > + List *children;
Hosoya-san,
Thanks for the new patch.
On Wed, Feb 26, 2020 at 11:33 AM yuzuko wrote:
> Attach the v5 patch. In this patch, pgstat_report_analyze() always reports 0
> as
> msg.m_live_tuples and m_dead_tuples when the relation is partitioned.
Some comments:
+ * PgStat_MsgPartAnalyzeSen
Hello Yuzuko,
> + * Report ANALYZE to the stats collector, too. If the table is a
> + * partition, report changes_since_analyze of its parent because
> + * autovacuum process for partitioned tables needs it. Reset the
> + * changes_since_analyze counter only if we analyzed al
On Wed, 26 Feb 2020 at 11:33, yuzuko wrote:
>
> Hi,
>
> Thanks for reviewing the patch.
>
> > > We can make it work correctly but I think perhaps we can skip updating
> > > statistics values of partitioned tables other than n_mod_since_analyze
> > > as the first step. Because if we support also n_
Hi,
Thanks for reviewing the patch.
> > We can make it work correctly but I think perhaps we can skip updating
> > statistics values of partitioned tables other than n_mod_since_analyze
> > as the first step. Because if we support also n_live_tup and
> > n_dead_tup, user might get confused that o
On Fri, Feb 21, 2020 at 4:47 PM Masahiko Sawada
wrote:
> Thank you for updating the patch. I tested v4 patch.
>
> After analyze or autoanalyze on partitioned table n_live_tup and
> n_dead_tup are updated. However, TRUNCATE and VACUUM on the
> partitioned table don't change these values until invok
On Fri, 21 Feb 2020 at 15:14, yuzuko wrote:
>
> Hello Amit-san,
>
> Thanks for your comments.
>
> > * White-space noise in the diff (space used where tab is expected);
> > please check with git diff --check and fix.
> Fixed it.
>
> > * Names changes_tuples, m_changes_tuples should be changed_tuple
Hello Amit-san,
Thanks for your comments.
> * White-space noise in the diff (space used where tab is expected);
> please check with git diff --check and fix.
Fixed it.
> * Names changes_tuples, m_changes_tuples should be changed_tuples and
> m_changed_tuples, respectively?
Yes, I modified it.
>
On Thu, Feb 20, 2020 at 5:32 PM Amit Langote wrote:
> On Thu, Feb 20, 2020 at 4:50 PM Amit Langote wrote:
> > * I may be missing something, but why doesn't do_autovacuum() fetch a
> > partitioned table's entry from pgstat instead of fetching that for
> > individual children and adding? That is, w
On Thu, Feb 20, 2020 at 4:50 PM Amit Langote wrote:
> * I may be missing something, but why doesn't do_autovacuum() fetch a
> partitioned table's entry from pgstat instead of fetching that for
> individual children and adding? That is, why do we need to do the
> following:
>
> +/*
> +
Hosoya-san,
On Thu, Feb 20, 2020 at 3:34 PM yuzuko wrote:
> Attach the latest patch based on discussion in this thread.
>
> > > Yeah that is what I meant. In addition, adding partition's
> > > changes_since_analyze to its parent needs to be done recursively as
> > > the parent table could also be
Hello,
I'm sorry for the delay.
Attach the latest patch based on discussion in this thread.
> > Yeah that is what I meant. In addition, adding partition's
> > changes_since_analyze to its parent needs to be done recursively as
> > the parent table could also be a partitioned table.
>
> That's a g
On Sun, Feb 2, 2020 at 12:53 PM Masahiko Sawada
wrote:
> On Wed, 29 Jan 2020 at 17:56, Amit Langote wrote:
> > On Wed, Jan 29, 2020 at 11:29 AM yuzuko wrote:
> > > > How are you going to track changes_since_analyze of partitioned table?
> > > > It's just an idea but we can accumulate changes_sin
On Wed, 29 Jan 2020 at 17:56, Amit Langote wrote:
>
> On Wed, Jan 29, 2020 at 11:29 AM yuzuko wrote:
> > > Besides the complexity of
> > > getting that infrastructure in place, an important question is whether
> > > the current system of applying threshold and scale factor to
> > > changes_since_
On Wed, Jan 29, 2020 at 05:56:40PM +0900, Amit Langote wrote:
> Yes, we will need to first support those parameters on partitioned
> tables. Currently, you get:
>
> create table p (a int) partition by list (a) with
> (autovacuum_analyze_scale_factor=0);
> ERROR: unrecognized parameter "autovacuu
On Wed, Jan 29, 2020 at 11:29 AM yuzuko wrote:
> > Besides the complexity of
> > getting that infrastructure in place, an important question is whether
> > the current system of applying threshold and scale factor to
> > changes_since_analyze should be used as-is for inheritance parents
> > (parti
1 - 100 of 108 matches
Mail list logo
