Re: vacuumdb changes for stats import/export

2025-07-30 Thread Nathan Bossart
Committed. -- nathan

Re: vacuumdb changes for stats import/export

2025-07-30 Thread Jeff Davis
On Wed, 2025-07-30 at 12:21 -0500, Nathan Bossart wrote: > Here is what I have staged for commit. That's more clear to me. I also like that it shows that the options work well together, because that was not obvious before. Regards, Jeff Davis

Re: vacuumdb changes for stats import/export

2025-07-30 Thread Corey Huinker
> > > It seems like there is some support for adding "When used in conjunction > > with --analyze in stages..." to the beginning of the sentence. I'll give > > it another day or two for any further discussion before committing. > > Here is what I have staged for commit. > +1

Re: vacuumdb changes for stats import/export

2025-07-30 Thread Nathan Bossart
On Mon, Jul 28, 2025 at 01:46:34PM -0500, Nathan Bossart wrote: > On Mon, Jul 28, 2025 at 09:22:29AM -0700, David G. Johnston wrote: >> On Monday, July 28, 2025, Frédéric Yhuel wrote: >>> On 7/28/25 16:47, Nathan Bossart wrote: Obviously, this wording isn't clear enough. We might need to eit

Re: vacuumdb changes for stats import/export

2025-07-28 Thread Nathan Bossart
On Mon, Jul 28, 2025 at 09:22:29AM -0700, David G. Johnston wrote: > On Monday, July 28, 2025, Frédéric Yhuel wrote: >> On 7/28/25 16:47, Nathan Bossart wrote: >>> Obviously, this wording isn't clear enough. We might need to either >>> remove that sentence or add "When used in conjunction with >>

Re: vacuumdb changes for stats import/export

2025-07-28 Thread Nathan Bossart
On Mon, Jul 28, 2025 at 09:22:29AM -0700, David G. Johnston wrote: > Makes sense. This does beg the question - what happens if a column is left > with a lower statistics target than what would be applied during an > analyze, but one is present? I don´t see where the statistics target is > saved a

Re: vacuumdb changes for stats import/export

2025-07-28 Thread David G. Johnston
On Monday, July 28, 2025, Frédéric Yhuel wrote: > > > On 7/28/25 16:47, Nathan Bossart wrote: > >> I can't remember who wrote this line, but it was borrowed from the >> --analyze-in-stages description. The point is that if you use >> --analyze-in-stages without --missing-stats-only, there will b

Re: vacuumdb changes for stats import/export

2025-07-28 Thread Frédéric Yhuel
On 7/28/25 16:47, Nathan Bossart wrote: I can't remember who wrote this line, but it was borrowed from the --analyze-in-stages description. The point is that if you use --analyze-in-stages without --missing-stats-only, there will be a period where existing statistics will be replaced with one

Re: vacuumdb changes for stats import/export

2025-07-28 Thread Nathan Bossart
On Sun, Jul 27, 2025 at 12:46:44PM +0200, Frédéric Yhuel wrote: > Then it seems very unlikely that the query optimizer's choices become > transiently worse because of that, doesn't it? and this shouldn't be used to > justify this option, IMHO. I can't remember who wrote this line, but it was borro

Re: vacuumdb changes for stats import/export

2025-07-27 Thread Frédéric Yhuel
On 7/26/25 21:38, Corey Huinker wrote: It is transactional, mostly. The attribute stats for the table being analyzed and all attribute stats for the dependent indexes that have at least one expression column, plus extended stats objects on that table, will be replaced in one atomic opera

Re: vacuumdb changes for stats import/export

2025-07-26 Thread Corey Huinker
> > > > > > Statistics are transactional. Without this option specified are we > really > > removing them and commiting prior to computing and saving new ones? And > we are > > opposed to just changing this behavior and instead prefer to add an > option that > > at first glance seems like everyon

Re: vacuumdb changes for stats import/export

2025-07-26 Thread Bruce Momjian
On Sat, Jul 26, 2025 at 07:47:47AM -0700, David G. Johnston wrote: > On Sat, Jul 26, 2025, 07:23 Bruce Momjian wrote: > > On Wed, Jul 23, 2025 at 02:45:10PM +0200, Frédéric Yhuel wrote: > > > > > > On 3/18/25 22:37, Nathan Bossart wrote: > > > Committed with the following smal

Re: vacuumdb changes for stats import/export

2025-07-26 Thread David G. Johnston
On Sat, Jul 26, 2025, 07:23 Bruce Momjian wrote: > On Wed, Jul 23, 2025 at 02:45:10PM +0200, Frédéric Yhuel wrote: > > > > > > On 3/18/25 22:37, Nathan Bossart wrote: > > > Committed with the following small changes: > > > > Hi, I don't really understand this sentence in > > doc/src/sgml/ref/vacu

Re: vacuumdb changes for stats import/export

2025-07-26 Thread Bruce Momjian
On Wed, Jul 23, 2025 at 02:45:10PM +0200, Frédéric Yhuel wrote: > > > On 3/18/25 22:37, Nathan Bossart wrote: > > Committed with the following small changes: > > Hi, I don't really understand this sentence in > doc/src/sgml/ref/vacuumdb.sgml: > > > This option prevents vacuumdb from deleting ex

Re: vacuumdb changes for stats import/export

2025-07-23 Thread Frédéric Yhuel
On 3/18/25 22:37, Nathan Bossart wrote: Committed with the following small changes: Hi, I don't really understand this sentence in doc/src/sgml/ref/vacuumdb.sgml: > This option prevents vacuumdb from deleting existing statistics so that the query optimizer's choices do not become transie

Re: vacuumdb changes for stats import/export

2025-03-18 Thread Nathan Bossart
Committed with the following small changes: * I renamed the new option to --missing-stats-only, which I felt was more descriptive. * I moved the new tests to the main vacuumdb test file and interspersed some --analyze-only uses. * I fixed a couple of other things in the new parts of the cata

Re: vacuumdb changes for stats import/export

2025-03-17 Thread Corey Huinker
> > > While preparing this for commit, I noticed that the expression index part > of the query was disregarding attstattarget. To fix, I've modified that > part to look at the index's pg_attribute entries. > +1, should have been there all along.

Re: vacuumdb changes for stats import/export

2025-03-17 Thread Nathan Bossart
On Fri, Mar 14, 2025 at 02:05:30PM -0500, Nathan Bossart wrote: > If no feedback or objections materialize, I'm planning to commit these > early next week. While preparing this for commit, I noticed that the expression index part of the query was disregarding attstattarget. To fix, I've modified

Re: vacuumdb changes for stats import/export

2025-03-15 Thread John Naylor
On Fri, Mar 7, 2025 at 4:47 AM Nathan Bossart wrote: > > On Thu, Mar 06, 2025 at 06:30:59PM +0700, John Naylor wrote: > > IIUC correctly, pg_statistic doesn't store stats on itself, so this > > causes the query result to always contain pg_statistic -- does that > > get removed elsewhere? > > Good

Re: vacuumdb changes for stats import/export

2025-03-15 Thread John Naylor
On Wed, Mar 12, 2025 at 12:00 AM Nathan Bossart wrote: > > On Mon, Mar 10, 2025 at 10:08:49AM -0500, Nathan Bossart wrote: > > On Mon, Mar 10, 2025 at 12:35:22PM +0700, John Naylor wrote: > >> I have no further comments. > > > > Thanks. I'll give this a little more time for review before committi

Re: vacuumdb changes for stats import/export

2025-03-15 Thread Nathan Bossart
On Mon, Mar 10, 2025 at 12:35:22PM +0700, John Naylor wrote: > I have no further comments. Thanks. I'll give this a little more time for review before committing. We'll still need to update the recommendation in pg_upgrade's documentation. I'm going to keep that separate because the stats impor

Re: vacuumdb changes for stats import/export

2025-03-15 Thread Nathan Bossart
On Thu, Mar 06, 2025 at 06:30:59PM +0700, John Naylor wrote: > +This option can only be used in conjunction with > +--analyze-only and > --analyze-in-stages. > > + /* Prohibit --missing-only without --analyze-only or --analyze-in-stages */ > + if (vacopts.missing_only && !vacopts.a

Re: vacuumdb changes for stats import/export

2025-03-14 Thread Nathan Bossart
Out of curiosity, I generated many relations with the following command (stolen from [0]): do $$ begin for i in 1..10 loop execute format('create table t%s (f1 int unique, f2 int unique);', i); execute format('insert into t%s select x, x from generate_series(1,1000) x', i)

Re: vacuumdb changes for stats import/export

2025-03-12 Thread Nathan Bossart
On Wed, Mar 12, 2025 at 01:56:04PM +0700, John Naylor wrote: > The change here seems fine. My only quibble is that this sentence now > seems out of place: "Option --analyze-in-stages can be used to > generate minimal statistics quickly." I'm thinking we should make a > clearer separation for with a

Re: vacuumdb changes for stats import/export

2025-03-11 Thread Nathan Bossart
On Mon, Mar 10, 2025 at 10:08:49AM -0500, Nathan Bossart wrote: > On Mon, Mar 10, 2025 at 12:35:22PM +0700, John Naylor wrote: >> I have no further comments. > > Thanks. I'll give this a little more time for review before committing. I realized that we could limit the catalog query reuse to only

Re: vacuumdb changes for stats import/export

2025-03-06 Thread John Naylor
On Wed, Mar 5, 2025 at 12:13 AM Nathan Bossart wrote: > > On Tue, Mar 04, 2025 at 01:05:17PM +0700, John Naylor wrote: > > On Mon, Mar 3, 2025 at 11:21 PM Nathan Bossart > > wrote: > >> I did that in v3. I also tried to break up this comment into bullet points > >> for better separation and log

Re: vacuumdb changes for stats import/export

2025-03-04 Thread Nathan Bossart
On Tue, Mar 04, 2025 at 01:05:17PM +0700, John Naylor wrote: > On Mon, Mar 3, 2025 at 11:21 PM Nathan Bossart > wrote: >> I did that in v3. I also tried to break up this comment into bullet points >> for better separation and logical flow. > > That's much easier to follow, thanks. Thanks for l

Re: vacuumdb changes for stats import/export

2025-03-03 Thread John Naylor
On Mon, Mar 3, 2025 at 11:21 PM Nathan Bossart wrote: > > On Mon, Mar 03, 2025 at 05:58:43PM +0700, John Naylor wrote: > True. One small thing we could do is to require "found_objs" (the double > pointer) to always be non-NULL, but that just compels some callers to > provide otherwise-unused vari

Re: vacuumdb changes for stats import/export

2025-03-03 Thread Nathan Bossart
On Mon, Mar 03, 2025 at 05:58:43PM +0700, John Naylor wrote: > On Sat, Mar 1, 2025 at 3:42 AM Nathan Bossart > wrote: >> On Thu, Feb 27, 2025 at 04:36:04PM +0700, John Naylor wrote: >> > I had to read it several times before I noticed the difference between >> > "* found_objs" and "*found_objs".

Re: vacuumdb changes for stats import/export

2025-03-03 Thread John Naylor
On Sat, Mar 1, 2025 at 3:42 AM Nathan Bossart wrote: > > On Thu, Feb 27, 2025 at 04:36:04PM +0700, John Naylor wrote: > > I had to read it several times before I noticed the difference between > > "* found_objs" and "*found_objs". Maybe some extra spacing and breaks > > would help, or other reorg

Re: vacuumdb changes for stats import/export

2025-02-28 Thread Nathan Bossart
On Thu, Feb 27, 2025 at 04:36:04PM +0700, John Naylor wrote: > On Wed, Feb 5, 2025 at 4:44 AM Nathan Bossart > wrote: >> [v2] > > I started looking just at 0001, and it seems like a fairly > straightforward rearrangement. Thanks for taking a look. > I found this comment quite hard to read: >

Re: vacuumdb changes for stats import/export

2025-02-27 Thread John Naylor
On Wed, Feb 5, 2025 at 4:44 AM Nathan Bossart wrote: > [v2] I started looking just at 0001, and it seems like a fairly straightforward rearrangement. I found this comment quite hard to read: + * 'found_objs' should be a fully qualified list of objects to process, as + * returned by a previous ca

Re: vacuumdb changes for stats import/export

2025-02-04 Thread Nathan Bossart
I had the opportunity to bring this patch set up for discussion at the developer meeting at FOSDEM PGDay last week [0]. There seemed to be a strong consensus that the idea of a "missing only" mode for vacuumdb's analyze options was useful (especially so if the extended stats piece of the stats imp

Re: vacuumdb changes for stats import/export

2025-01-24 Thread Masahiko Sawada
On Fri, Jan 24, 2025 at 7:44 AM Nathan Bossart wrote: > > On Mon, Jan 06, 2025 at 03:27:18PM -0600, Nathan Bossart wrote: > > On Mon, Dec 30, 2024 at 03:45:03PM -0500, Bruce Momjian wrote: > >> On Mon, Dec 30, 2024 at 12:02:47PM -0800, Jeff Davis wrote: > >>> I suggest that we make a new thread ab

Re: vacuumdb changes for stats import/export

2025-01-24 Thread Corey Huinker
> > Thoughts? > I don't have anything to add to what Nathan said, but thought I should say so since this thread was broken off from my earlier thread. Eagerly awaiting feedback.

vacuumdb changes for stats import/export

2025-01-24 Thread Nathan Bossart
On Mon, Jan 06, 2025 at 03:27:18PM -0600, Nathan Bossart wrote: > On Mon, Dec 30, 2024 at 03:45:03PM -0500, Bruce Momjian wrote: >> On Mon, Dec 30, 2024 at 12:02:47PM -0800, Jeff Davis wrote: >>> I suggest that we make a new thread about the vacuumdb changes and >>> focus this thread and patch seri