Re: Using multiple extended statistics for estimates

2020-01-12 Thread Tomas Vondra
Hi, I've pushed these two improvements after some minor improvements, mostly to comments. I ended up not using the extra tests, as it wasn't clear to me it's worth the extra duration. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remot

Re: Using multiple extended statistics for estimates

2019-12-09 Thread Mark Dilger
On 12/9/19 2:00 PM, Tomas Vondra wrote: These look good to me.  I added extra tests (not included in this email) to verify the code on more interesting test cases, such as partitioned tables and within joins.  Your test cases are pretty trivial, just being selects from a single table. Addin

Re: Using multiple extended statistics for estimates

2019-12-09 Thread Tomas Vondra
On Mon, Dec 09, 2019 at 11:56:39AM -0800, Mark Dilger wrote: On 12/5/19 9:51 AM, Tomas Vondra wrote: On Thu, Dec 05, 2019 at 06:15:54PM +0100, Tomas Vondra wrote: On Sun, Dec 01, 2019 at 08:08:58PM +0100, Tomas Vondra wrote: On Sat, Nov 30, 2019 at 03:01:31PM -0800, Mark Dilger wrote: Are

Re: Using multiple extended statistics for estimates

2019-12-09 Thread Mark Dilger
On 12/5/19 9:51 AM, Tomas Vondra wrote: On Thu, Dec 05, 2019 at 06:15:54PM +0100, Tomas Vondra wrote: On Sun, Dec 01, 2019 at 08:08:58PM +0100, Tomas Vondra wrote: On Sat, Nov 30, 2019 at 03:01:31PM -0800, Mark Dilger wrote: Are you planning to submit a revised patch for this? Yes, I'll

Re: Using multiple extended statistics for estimates

2019-12-05 Thread Tomas Vondra
On Thu, Dec 05, 2019 at 06:15:54PM +0100, Tomas Vondra wrote: On Sun, Dec 01, 2019 at 08:08:58PM +0100, Tomas Vondra wrote: On Sat, Nov 30, 2019 at 03:01:31PM -0800, Mark Dilger wrote: Are you planning to submit a revised patch for this? Yes, I'll submit a rebased version of this patch shor

Re: Using multiple extended statistics for estimates

2019-12-05 Thread Tomas Vondra
On Sun, Dec 01, 2019 at 08:08:58PM +0100, Tomas Vondra wrote: On Sat, Nov 30, 2019 at 03:01:31PM -0800, Mark Dilger wrote: Are you planning to submit a revised patch for this? Yes, I'll submit a rebased version of this patch shortly. I got broken because of the recent fix in choose_best_stat

Re: Using multiple extended statistics for estimates

2019-12-01 Thread Tomas Vondra
On Sat, Nov 30, 2019 at 03:01:31PM -0800, Mark Dilger wrote: Are you planning to submit a revised patch for this? Yes, I'll submit a rebased version of this patch shortly. I got broken because of the recent fix in choose_best_statistics, shouldn't take long to update the patch. I do have a co

Re: Using multiple extended statistics for estimates

2019-11-30 Thread Mark Dilger
On 11/14/19 12:04 PM, Tomas Vondra wrote: On Thu, Nov 14, 2019 at 10:23:44AM -0800, Mark Dilger wrote: On 11/14/19 7:55 AM, Tomas Vondra wrote: On Wed, Nov 13, 2019 at 10:04:36AM -0800, Mark Dilger wrote: On 11/13/19 7:28 AM, Tomas Vondra wrote: Hi, here's an updated patch, with some

Re: Using multiple extended statistics for estimates

2019-11-14 Thread Tomas Vondra
On Thu, Nov 14, 2019 at 01:17:02PM -0800, Mark Dilger wrote: On 11/14/19 12:04 PM, Tomas Vondra wrote: Aha, I think I understand now - thanks for the explanation. You're right the comment is trying to explain why just taking the last clause for a given attnum is fine. I'll try to make the comm

Re: Using multiple extended statistics for estimates

2019-11-14 Thread Tomas Vondra
On Thu, Nov 14, 2019 at 03:16:04PM -0500, Tom Lane wrote: Tomas Vondra writes: For the case with equal Const values that should be mostly obvious, i.e. "a=1 AND a=1 AND a=1" has the same selectivity as "a=1". The case with different Const values is harder, unfortunately. It might seem obviou

Re: Using multiple extended statistics for estimates

2019-11-14 Thread Mark Dilger
On 11/14/19 12:04 PM, Tomas Vondra wrote: Aha, I think I understand now - thanks for the explanation. You're right the comment is trying to explain why just taking the last clause for a given attnum is fine. I'll try to make the comment clearer. For the case with equal Const values that shoul

Re: Using multiple extended statistics for estimates

2019-11-14 Thread Tom Lane
Tomas Vondra writes: > For the case with equal Const values that should be mostly obvious, i.e. > "a=1 AND a=1 AND a=1" has the same selectivity as "a=1". > The case with different Const values is harder, unfortunately. It might > seem obvious that "a=1 AND a=2" means there are no matching rows,

Re: Using multiple extended statistics for estimates

2019-11-14 Thread Tomas Vondra
On Thu, Nov 14, 2019 at 10:23:44AM -0800, Mark Dilger wrote: On 11/14/19 7:55 AM, Tomas Vondra wrote: On Wed, Nov 13, 2019 at 10:04:36AM -0800, Mark Dilger wrote: On 11/13/19 7:28 AM, Tomas Vondra wrote: Hi, here's an updated patch, with some minor tweaks based on the review and added tes

Re: Using multiple extended statistics for estimates

2019-11-14 Thread Mark Dilger
On 11/14/19 7:55 AM, Tomas Vondra wrote: On Wed, Nov 13, 2019 at 10:04:36AM -0800, Mark Dilger wrote: On 11/13/19 7:28 AM, Tomas Vondra wrote: Hi, here's an updated patch, with some minor tweaks based on the review and added tests (I ended up reworking those a bit, to make them more like

Re: Using multiple extended statistics for estimates

2019-11-14 Thread Tomas Vondra
On Wed, Nov 13, 2019 at 10:04:36AM -0800, Mark Dilger wrote: On 11/13/19 7:28 AM, Tomas Vondra wrote: Hi, here's an updated patch, with some minor tweaks based on the review and added tests (I ended up reworking those a bit, to make them more like the existing ones). Thanks, Tomas, for the

Re: Using multiple extended statistics for estimates

2019-11-13 Thread Mark Dilger
On 11/13/19 7:28 AM, Tomas Vondra wrote: Hi, here's an updated patch, with some minor tweaks based on the review and added tests (I ended up reworking those a bit, to make them more like the existing ones). Thanks, Tomas, for the new patch set! Attached are my review comments so far, in the

Re: Using multiple extended statistics for estimates

2019-11-13 Thread Tomas Vondra
Hi, here's an updated patch, with some minor tweaks based on the review and added tests (I ended up reworking those a bit, to make them more like the existing ones). There's also a new piece, dealing with functional dependencies. Until now we did the same thing as for MCV lists - we picketd the

Re: Using multiple extended statistics for estimates

2019-11-10 Thread Tomas Vondra
On Sat, Nov 09, 2019 at 02:32:27PM -0800, Mark Dilger wrote: On 11/9/19 12:33 PM, Mark Dilger wrote: On 11/6/19 11:58 AM, Tomas Vondra wrote: On Wed, Nov 06, 2019 at 08:54:40PM +0100, Tomas Vondra wrote: On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10 intr

Re: Using multiple extended statistics for estimates

2019-11-10 Thread Tomas Vondra
On Sat, Nov 09, 2019 at 12:33:05PM -0800, Mark Dilger wrote: On 11/6/19 11:58 AM, Tomas Vondra wrote: On Wed, Nov 06, 2019 at 08:54:40PM +0100, Tomas Vondra wrote: On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10 introduced extended statistics, allowing us to

Re: Using multiple extended statistics for estimates

2019-11-09 Thread Mark Dilger
On 11/9/19 12:33 PM, Mark Dilger wrote: On 11/6/19 11:58 AM, Tomas Vondra wrote: On Wed, Nov 06, 2019 at 08:54:40PM +0100, Tomas Vondra wrote: On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10 introduced extended statistics, allowing us to consider correlatio

Re: Using multiple extended statistics for estimates

2019-11-09 Thread Mark Dilger
On 11/6/19 11:58 AM, Tomas Vondra wrote: On Wed, Nov 06, 2019 at 08:54:40PM +0100, Tomas Vondra wrote: On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10 introduced extended statistics, allowing us to consider correlation between columns to improve estimates, and

Re: Using multiple extended statistics for estimates

2019-11-07 Thread Tomas Vondra
On Thu, Nov 07, 2019 at 01:38:20PM +0900, Kyotaro Horiguchi wrote: Hello. At Wed, 6 Nov 2019 20:58:49 +0100, Tomas Vondra wrote in >Here is a slightly polished v2 of the patch, the main difference being >that computing clause_attnums was moved to a separate function. > This time with the att

Re: Using multiple extended statistics for estimates

2019-11-06 Thread Kyotaro Horiguchi
Hello. At Wed, 6 Nov 2019 20:58:49 +0100, Tomas Vondra wrote in > >Here is a slightly polished v2 of the patch, the main difference being > >that computing clause_attnums was moved to a separate function. > > > > This time with the attachment ;-) This patch is a kind of straight-forward, whic

Re: Using multiple extended statistics for estimates

2019-11-06 Thread Tomas Vondra
On Wed, Nov 06, 2019 at 08:54:40PM +0100, Tomas Vondra wrote: On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10 introduced extended statistics, allowing us to consider correlation between columns to improve estimates, and PostgreSQL 12 added support for MCV statist

Re: Using multiple extended statistics for estimates

2019-11-06 Thread Tomas Vondra
On Mon, Oct 28, 2019 at 04:20:48PM +0100, Tomas Vondra wrote: Hi, PostgreSQL 10 introduced extended statistics, allowing us to consider correlation between columns to improve estimates, and PostgreSQL 12 added support for MCV statistics. But we still had the limitation that we only allowed using