Re: Add important info about ANALYZE after create Functional Index

2020-11-16 Thread Bruce Momjian
On Mon, Nov 16, 2020 at 11:59:03AM -0300, Álvaro Herrera wrote: > On 2020-Nov-12, Bruce Momjian wrote: > > > For new expression indexes, it is necessary to run > linkend="sql-analyze">ANALYZE or wait for > > the autovacuum daemon to analyze > > - the table to generate statistics abo

Re: Add important info about ANALYZE after create Functional Index

2020-11-16 Thread Alvaro Herrera
On 2020-Nov-16, Justin Pryzby wrote: > I see Alvaro already patched the first issue at bcbd77133. > > The problematic language was recently introduced, and I'd reported at: > https://www.postgresql.org/message-id/20201112211143.GL30691%40telsasoft.com > And Erik at: > https://www.postgresql.org/m

Re: Add important info about ANALYZE after create Functional Index

2020-11-16 Thread Alvaro Herrera
On 2020-Nov-12, Bruce Momjian wrote: > For new expression indexes, it is necessary to run linkend="sql-analyze">ANALYZE or wait for > the autovacuum daemon to analyze > - the table to generate statistics about new expression indexes. > + the table to generate statistics for these

Re: Add important info about ANALYZE after create Functional Index

2020-11-16 Thread Justin Pryzby
On Thu, Nov 12, 2020 at 06:01:02PM -0500, Bruce Momjian wrote: > On Thu, Nov 12, 2020 at 03:11:43PM -0600, Justin Pryzby wrote: > > I guess it should say "The system regularly ..." > > > > Also, the last sentence begins "For new expression indexes" and ends with > > "about new expression indexes",

Re: Add important info about ANALYZE after create Functional Index

2020-11-12 Thread Bruce Momjian
On Thu, Nov 12, 2020 at 03:11:43PM -0600, Justin Pryzby wrote: > I guess it should say "The system regularly ..." > > Also, the last sentence begins "For new expression indexes" and ends with > "about new expression indexes", which I guess could instead say "about the > expressions". How is this

Re: Add important info about ANALYZE after create Functional Index

2020-11-12 Thread Justin Pryzby
On Mon, Nov 09, 2020 at 06:27:20PM -0500, Bruce Momjian wrote: > On Tue, Oct 27, 2020 at 12:12:00AM -0700, Nikolay Samokhvalov wrote: > > On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello > > wrote: > > > > Would be nice if add some information about it into our docs but not > > sure

Re: Add important info about ANALYZE after create Functional Index

2020-11-12 Thread Bruce Momjian
On Mon, Nov 9, 2020 at 08:35:46PM -0300, Fabrízio de Royes Mello wrote: > > > On Mon, 9 Nov 2020 at 20:27 Bruce Momjian wrote: > > > I see REINDEX CONCURRENTLY was fixed in head, but the docs didn't get > updated to mention the need to run ANALYZE or wait for autovacuum before > e

Re: Add important info about ANALYZE after create Functional Index

2020-11-09 Thread Fabrízio de Royes Mello
On Mon, 9 Nov 2020 at 20:27 Bruce Momjian wrote: > > I see REINDEX CONCURRENTLY was fixed in head, but the docs didn't get > updated to mention the need to run ANALYZE or wait for autovacuum before > expression indexes can be fully used by the optimizer. Instead of > putting this mention in the

Re: Add important info about ANALYZE after create Functional Index

2020-11-09 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 12:12:00AM -0700, Nikolay Samokhvalov wrote: > On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < > fabriziome...@gmail.com> wrote: > > Would be nice if add some information about it into our docs but not sure > where. I'm thinking about: > - doc/src/sgml

Re: Add important info about ANALYZE after create Functional Index

2020-11-01 Thread Fabrízio de Royes Mello
On Sun, 1 Nov 2020 at 09:29 Michael Paquier wrote: > On Sun, Nov 01, 2020 at 09:23:44AM +0900, Michael Paquier wrote: > > By doing so, there is no need to include pg_statistic.h in index.c. > > Except that, the logic looks fine at quick glance. In the long-term, > > I also think that it would ma

Re: Add important info about ANALYZE after create Functional Index

2020-11-01 Thread Michael Paquier
On Sun, Nov 01, 2020 at 09:23:44AM +0900, Michael Paquier wrote: > By doing so, there is no need to include pg_statistic.h in index.c. > Except that, the logic looks fine at quick glance. In the long-term, > I also think that it would make sense to move both routnes out of > heap.c into a separate

Re: Add important info about ANALYZE after create Functional Index

2020-10-31 Thread Justin Pryzby
On Sun, Nov 01, 2020 at 10:11:06AM +0900, Michael Paquier wrote: > On Fri, Oct 30, 2020 at 10:30:13PM -0500, Justin Pryzby wrote: > > (I'm quoting from the commit message of the patch I wrote, which is same as > > your patch). > > (I may have missed something, but you did not send a patch, right?)

Re: Add important info about ANALYZE after create Functional Index

2020-10-31 Thread Michael Paquier
On Fri, Oct 30, 2020 at 10:30:13PM -0500, Justin Pryzby wrote: > (I'm quoting from the commit message of the patch I wrote, which is same as > your patch). (I may have missed something, but you did not send a patch, right?) -- Michael signature.asc Description: PGP signature

Re: Add important info about ANALYZE after create Functional Index

2020-10-31 Thread Michael Paquier
On Sat, Oct 31, 2020 at 07:56:33PM -0300, Fabrízio de Royes Mello wrote: > Even if we won't use it now, IMHO it is more legible to separate this > responsibility into its own CopyStatistics function as attached. By doing so, there is no need to include pg_statistic.h in index.c. Except that, the l

Re: Add important info about ANALYZE after create Functional Index

2020-10-31 Thread Fabrízio de Royes Mello
On Fri, Oct 30, 2020 at 3:22 AM Michael Paquier wrote: > > And in spirit, it is possible to address this issue with the patch > attached which copies the set of stats from the old to the new index. Did some tests and everything went ok... some comments below! > For a non-concurrent REINDEX, this

Re: Add important info about ANALYZE after create Functional Index

2020-10-30 Thread Justin Pryzby
On Fri, Oct 30, 2020 at 03:22:52PM +0900, Michael Paquier wrote: > On Thu, Oct 29, 2020 at 10:59:52AM +0900, Michael Paquier wrote: > > REINDEX CONCURRENTLY is by design wanted to provide an experience > > transparent to the user similar to what a plain REINDEX would do, at > > least that's the ide

Re: Add important info about ANALYZE after create Functional Index

2020-10-29 Thread Michael Paquier
On Thu, Oct 29, 2020 at 10:59:52AM +0900, Michael Paquier wrote: > REINDEX CONCURRENTLY is by design wanted to provide an experience > transparent to the user similar to what a plain REINDEX would do, at > least that's the idea behind it, so.. This qualifies as a bug to me, > in spirit. And in sp

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Michael Paquier
On Thu, Oct 29, 2020 at 12:02:11AM +0100, Tomas Vondra wrote: > On Wed, Oct 28, 2020 at 05:43:08PM -0300, Fabrízio de Royes Mello wrote: >> 2) REINDEX CONCURRENTLY does not keep statistics (pg_statistc) like a >> regular REINDEX for indexes using expressions and to me it's a bug. Michael >> pointed

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 05:43:08PM -0300, Fabrízio de Royes Mello wrote: On Wed, Oct 28, 2020 at 4:35 PM Tomas Vondra wrote: I don't think anyone proposed to do this through autovacuum. There was a reference to auto-analyze but I think that was meant as 'run analyze automatically.' Which would

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Fabrízio de Royes Mello
On Wed, Oct 28, 2020 at 4:35 PM Tomas Vondra wrote: > > I don't think anyone proposed to do this through autovacuum. There was a > reference to auto-analyze but I think that was meant as 'run analyze > automatically.' Which would work in transactions just fine, I think. > Maybe I was not very cle

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:18:52PM -0400, Tom Lane wrote: Tomas Vondra writes: On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: Given how simple the manual workaround is not having it be manual seems like it would be safe and straight-forward to implement. Maybe, but I wou

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tom Lane
Tomas Vondra writes: > On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: >> Given how simple the manual workaround is not having it be manual seems >> like it would be safe and straight-forward to implement. > Maybe, but I wouldn't be surprised if it was actually a bit trickier i

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 12:05 PM Tom Lane wrote: > This doesn't seem clearly different from any other situation where > auto-analyze doesn't react fast enough to suit you. > I would not > call it a bug, at least not without a wholesale redefinition of > how auto-analyze is supposed to work.

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:05:39PM -0400, Tom Lane wrote: Tomas Vondra writes: On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze side-effect ju

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: On Wed, Oct 28, 2020 at 11:55 AM Tomas Vondra wrote: I agree the lack of stats may be quite annoying and cause issues, but my guess is the chances of backpatching such change are about 0.01%. We have a usable 'workaround' f

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Mon, Oct 26, 2020 at 9:44 PM Nikolay Samokhvalov wrote: > On Mon, Oct 26, 2020 at 7:03 PM David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> On Monday, October 26, 2020, Nikolay Samokhvalov >> wrote: >>> >>> Although, this triggers a question – should ANALYZE be automated in, >>> s

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tom Lane
Tomas Vondra writes: > On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: >> It would seem preferable to call the lack of auto-analyzing after these >> operations a bug and back-patch a fix that injects an analyze side-effect >> just before their completion. It doesn't have to be

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 11:55 AM Tomas Vondra wrote: > I agree the lack of stats may be quite annoying and cause issues, but my > guess is the chances of backpatching such change are about 0.01%. We > have a usable 'workaround' for this - manual analyze. > My guess is that it wouldn't be too

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: Hi all, As you all already know Postgres supports functions in index expressions (marked as immutable ofc) and for this special index the

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: On Mon, Oct 26, 2020 at 7:46 PM David G. Johnston < david.g.johns...@gmail.com> wrote: It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Fabrízio de Royes Mello
On Wed, Oct 28, 2020 at 2:15 AM Michael Paquier wrote: > > On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: > > When we create a new table or index they will not have statistics until an > > ANALYZE happens. This is the default behaviour and I think is not a big > > problem

Re: Add important info about ANALYZE after create Functional Index

2020-10-27 Thread Michael Paquier
On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: > When we create a new table or index they will not have statistics until an > ANALYZE happens. This is the default behaviour and I think is not a big > problem here, but we need to add some note on docs about the need of > st

Re: Add important info about ANALYZE after create Functional Index

2020-10-27 Thread Fabrízio de Royes Mello
On Tue, Oct 27, 2020 at 4:12 AM Nikolay Samokhvalov wrote: > > On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: >> >> Would be nice if add some information about it into our docs but not sure where. I'm thinking about: >> - doc/src/sgml/ref/create_index.sgm

Re: Add important info about ANALYZE after create Functional Index

2020-10-27 Thread Fabrízio de Royes Mello
On Mon, Oct 26, 2020 at 7:46 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > > It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze side-effect just before their completion. It doesn't have to be smart eit

Re: Add important info about ANALYZE after create Functional Index

2020-10-27 Thread Nikolay Samokhvalov
On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > Would be nice if add some information about it into our docs but not sure > where. I'm thinking about: > - doc/src/sgml/ref/create_index.sgml > - doc/src/sgml/maintenance.sgml (routine-reindex) > Attaching

Re: Add important info about ANALYZE after create Functional Index

2020-10-26 Thread Nikolay Samokhvalov
On Mon, Oct 26, 2020 at 7:03 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Monday, October 26, 2020, Nikolay Samokhvalov > wrote: >> >> Although, this triggers a question – should ANALYZE be automated in, say, >> pg_restore as well? >> > > Independent concern. > It's the same cl

Re: Add important info about ANALYZE after create Functional Index

2020-10-26 Thread David G. Johnston
On Monday, October 26, 2020, Nikolay Samokhvalov wrote: > > > Although, this triggers a question – should ANALYZE be automated in, say, > pg_restore as well? > Independent concern. > > And another question: how ANALYZE needs to be run? If it's under the > user's control, there is an option to u

Re: Add important info about ANALYZE after create Functional Index

2020-10-26 Thread Nikolay Samokhvalov
On Mon, Oct 26, 2020 at 3:46 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > It would seem preferable to call the lack of auto-analyzing after these > operations a bug and back-patch a fix that injects an analyze side-effect > just before their completion. It doesn't have to be smart

Re: Add important info about ANALYZE after create Functional Index

2020-10-26 Thread David G. Johnston
On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > Hi all, > > As you all already know Postgres supports functions in index expressions > (marked as immutable ofc) and for this special index the ANALYZE command > creates some statistics (new pg_statistic en