Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-07-06 Thread David Rowley
On Tue, 6 Jul 2021 at 22:39, David Rowley wrote: > If anyone feels differently, please let me know in the next couple of > days. Otherwise, I plan on taking a final look and pushing it soon. After doing some very minor adjustments, I pushed this. (29f45e299). Thanks to James and Zhihong for revi

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-07-06 Thread David Rowley
I've been looking at the NOT IN hashing patch again and after making a few minor tweaks I think it's pretty much ready to go. If anyone feels differently, please let me know in the next couple of days. Otherwise, I plan on taking a final look and pushing it soon. David v5-0001-Use-hash-table-to

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-22 Thread David Rowley
On Sat, 8 May 2021 at 20:29, David Rowley wrote: > > On Sat, 8 May 2021 at 20:17, Zhihong Yu wrote: > > + if (!OidIsValid(saop->negfuncid)) > > + record_plan_function_dependency(root, saop->hashfuncid); > > > > Is there a typo in the second line ? (root, saop->negfuncid) > > Yeah,

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-08 Thread David Rowley
On Sat, 8 May 2021 at 20:17, Zhihong Yu wrote: > + if (!OidIsValid(saop->negfuncid)) > + record_plan_function_dependency(root, saop->hashfuncid); > > Is there a typo in the second line ? (root, saop->negfuncid) Yeah, that's a mistake. Thanks for checking it. David

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-08 Thread Zhihong Yu
On Fri, May 7, 2021 at 9:50 PM David Rowley wrote: > On Sat, 8 May 2021 at 14:04, David Rowley wrote: > > I'm not opposed to adding some new field if that's what it takes. I'd > > imagine the new field will be something like negfuncid which will be > > InvalidOid unless the hash function is set

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 14:04, David Rowley wrote: > I'm not opposed to adding some new field if that's what it takes. I'd > imagine the new field will be something like negfuncid which will be > InvalidOid unless the hash function is set and useOr == false Just while this is still swapped into ma

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 13:37, James Coleman wrote: > > On Fri, May 7, 2021 at 9:16 PM Tom Lane wrote: > > I don't immediately see why you can't add an "invert" boolean flag to > > ScalarArrayOpExpr and let the executor machinery deal with this. That'd > > have the advantage of not having to depen

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-07 Thread James Coleman
On Fri, May 7, 2021 at 8:38 PM David Rowley wrote: > > It's important to think of other cases, I just don't think there's any > need to do anything for that one. Remember that we have the > restriction of requiring a set of Consts, so for that case to be met, > someone would have to write somethi

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-07 Thread James Coleman
On Fri, May 7, 2021 at 9:16 PM Tom Lane wrote: > > David Rowley writes: > > On Sat, 8 May 2021 at 09:15, James Coleman wrote: > >> On Sat, Apr 24, 2021 at 6:25 AM David Rowley wrote: > >>> I'm a bit undecided if it's safe to set the opfuncid to the negator > >>> function. If anything were to s

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-07 Thread Tom Lane
David Rowley writes: > On Sat, 8 May 2021 at 09:15, James Coleman wrote: >> On Sat, Apr 24, 2021 at 6:25 AM David Rowley wrote: >>> I'm a bit undecided if it's safe to set the opfuncid to the negator >>> function. If anything were to set that again based on the opno then >>> it would likely set

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 09:15, James Coleman wrote: > > On Sat, Apr 24, 2021 at 6:25 AM David Rowley wrote: > > I'm a bit undecided if it's safe to set the opfuncid to the negator > > function. If anything were to set that again based on the opno then > > it would likely set it to the wrong thing.

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-05-07 Thread James Coleman
On Sat, Apr 24, 2021 at 6:25 AM David Rowley wrote: > > On Wed, 14 Apr 2021 at 05:40, James Coleman wrote: > > ...and here's a draft patch. I can take this to a new thread if you'd > > prefer; the one here already got committed, on the other hand this is > > pretty strongly linked to this discuss

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-24 Thread David Rowley
On Wed, 14 Apr 2021 at 05:40, James Coleman wrote: > ...and here's a draft patch. I can take this to a new thread if you'd > prefer; the one here already got committed, on the other hand this is > pretty strongly linked to this discussion, so I figured it made sense > to post it here. I only glan

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-13 Thread James Coleman
On Mon, Apr 12, 2021 at 10:07 PM James Coleman wrote: > > On Mon, Apr 12, 2021 at 7:49 PM David Rowley wrote: > > > > On Tue, 13 Apr 2021 at 11:42, Tom Lane wrote: > > > > > > David Rowley writes: > > > > I realised when working on something unrelated last night that we can > > > > also do hash

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-12 Thread James Coleman
On Mon, Apr 12, 2021 at 7:49 PM David Rowley wrote: > > On Tue, 13 Apr 2021 at 11:42, Tom Lane wrote: > > > > David Rowley writes: > > > I realised when working on something unrelated last night that we can > > > also do hash lookups for NOT IN too. > > > > ... and still get the behavior right f

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-12 Thread David Rowley
On Tue, 13 Apr 2021 at 11:42, Tom Lane wrote: > > David Rowley writes: > > I realised when working on something unrelated last night that we can > > also do hash lookups for NOT IN too. > > ... and still get the behavior right for nulls? Yeah, it will. There are already some special cases for NU

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-12 Thread Tom Lane
David Rowley writes: > I realised when working on something unrelated last night that we can > also do hash lookups for NOT IN too. ... and still get the behavior right for nulls? regards, tom lane

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-12 Thread David Rowley
On Fri, 9 Apr 2021 at 00:00, David Rowley wrote: > I push this with some minor cleanup from the v6 patch I posted earlier. I realised when working on something unrelated last night that we can also do hash lookups for NOT IN too. We'd just need to check if the operator's negator operator is hash

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-12 Thread David Rowley
On Sun, 11 Apr 2021 at 10:38, Tomas Vondra wrote: > I wonder what's the relationship between the length of the IN list and > the minimum number of rows needed for the hash to start winning. I made the attached spreadsheet which demonstrates the crossover point using the costs that I coded into co

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-10 Thread Tomas Vondra
On 4/11/21 12:03 AM, David Rowley wrote: > On Sat, 10 Apr 2021 at 10:32, Tomas Vondra > wrote: >> But I think the puzzle is not so much about v5 vs v6, but more about v5 >> vs. master. I still don't understand how v5 managed to be faster than >> master, but maybe I'm missing something. > > Wel

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-10 Thread David Rowley
On Sat, 10 Apr 2021 at 10:32, Tomas Vondra wrote: > But I think the puzzle is not so much about v5 vs v6, but more about v5 > vs. master. I still don't understand how v5 managed to be faster than > master, but maybe I'm missing something. Well, v5 wrapped ScalarArrayOpExpr inside HashedScalarArra

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-09 Thread Tomas Vondra
On 4/9/21 1:21 AM, David Rowley wrote: > On Fri, 9 Apr 2021 at 09:32, Tomas Vondra > wrote: >> >> I ran the same set of benchmarks on the v6, which I think should be >> mostly identical to what was committed. I extended the test a bit to >> test table with 0, 1, 5 and 1000 rows, and also with

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread David Rowley
On Fri, 9 Apr 2021 at 09:32, Tomas Vondra wrote: > > I ran the same set of benchmarks on the v6, which I think should be > mostly identical to what was committed. I extended the test a bit to > test table with 0, 1, 5 and 1000 rows, and also with int and text > values, to see how it works with mor

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread Tomas Vondra
On 4/8/21 2:00 PM, David Rowley wrote: > On Thu, 8 Apr 2021 at 22:54, David Rowley wrote: >> >> I think the changes in the patch are fairly isolated and the test >> coverage is now pretty good. I'm planning on looking at the patch >> again now and will consider pushing it for PG14. > > I push th

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, James Coleman wrote: > On Thu, Apr 8, 2021 at 1:04 PM Alvaro Herrera wrote: > > > > On 2021-Apr-08, James Coleman wrote: > > > > > I assume proper procedure for the CF entry is to move it into the > > > current CF and then mark it as committed, however I don't know how (or > > > d

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread James Coleman
On Thu, Apr 8, 2021 at 1:04 PM Alvaro Herrera wrote: > > On 2021-Apr-08, James Coleman wrote: > > > I assume proper procedure for the CF entry is to move it into the > > current CF and then mark it as committed, however I don't know how (or > > don't have permissions?) to move it into the current

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, James Coleman wrote: > I assume proper procedure for the CF entry is to move it into the > current CF and then mark it as committed, however I don't know how (or > don't have permissions?) to move it into the current CF. How does one > go about doing that? > > Here's the entry: ht

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread James Coleman
On Thu, Apr 8, 2021 at 8:01 AM David Rowley wrote: > > On Thu, 8 Apr 2021 at 22:54, David Rowley wrote: > > > > I think the changes in the patch are fairly isolated and the test > > coverage is now pretty good. I'm planning on looking at the patch > > again now and will consider pushing it for P

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread David Rowley
On Thu, 8 Apr 2021 at 22:54, David Rowley wrote: > > I think the changes in the patch are fairly isolated and the test > coverage is now pretty good. I'm planning on looking at the patch > again now and will consider pushing it for PG14. I push this with some minor cleanup from the v6 patch I po

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread David Rowley
On Thu, 8 Apr 2021 at 18:50, David Rowley wrote: > I've not done any further performance tests yet but will start those now. I ran a set of tests on this: select * from a where a in( < 1 to 10 > ); and select * from a where a in( < 1 to 100 > ); the table "a" is just an empty table with a singl

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-07 Thread David Rowley
On Thu, 8 Apr 2021 at 05:54, Tomas Vondra wrote: > I only ran that with a single client, the machine only has 4 cores and > this should not be related to concurrency, so 1 client seems fine. The > average of 10 runs, 15 seconds each look like this: Thanks for running these tests. The reason I a

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-07 Thread Tomas Vondra
On 4/6/21 5:58 AM, David Rowley wrote: > On Sat, 20 Mar 2021 at 09:41, James Coleman wrote: >> I've attached a cleaned up patch. Last CF it was listed in is >> https://commitfest.postgresql.org/29/2542/ -- what's the appropriate >> step to take here given it's an already existing patch, but not

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-06 Thread James Coleman
On Mon, Apr 5, 2021 at 11:58 PM David Rowley wrote: > > On Sat, 20 Mar 2021 at 09:41, James Coleman wrote: > > I've attached a cleaned up patch. Last CF it was listed in is > > https://commitfest.postgresql.org/29/2542/ -- what's the appropriate > > step to take here given it's an already existin

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-05 Thread David Rowley
On Sat, 20 Mar 2021 at 09:41, James Coleman wrote: > I've attached a cleaned up patch. Last CF it was listed in is > https://commitfest.postgresql.org/29/2542/ -- what's the appropriate > step to take here given it's an already existing patch, but not yet > moved into recent CFs? I had a look at

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-03-19 Thread James Coleman
On Fri, Sep 11, 2020 at 5:11 PM James Coleman wrote: > > On Tue, Sep 8, 2020 at 4:37 PM Heikki Linnakangas wrote: > > > > On 08/09/2020 22:25, James Coleman wrote: > > > On Wed, Aug 19, 2020 at 3:16 AM Heikki Linnakangas > > > wrote: > > >> > > >> You could also apply the optimization for non-C

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-09-11 Thread James Coleman
On Tue, Sep 8, 2020 at 4:37 PM Heikki Linnakangas wrote: > > On 08/09/2020 22:25, James Coleman wrote: > > On Wed, Aug 19, 2020 at 3:16 AM Heikki Linnakangas wrote: > >> > >> You could also apply the optimization for non-Const expressions, as long > >> as they're stable (i.e. !contain_volatile_fu

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-09-08 Thread Heikki Linnakangas
On 08/09/2020 22:25, James Coleman wrote: On Wed, Aug 19, 2020 at 3:16 AM Heikki Linnakangas wrote: You could also apply the optimization for non-Const expressions, as long as they're stable (i.e. !contain_volatile_functions(expr)). Is that true? Don't we also have to worry about whether or

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-09-08 Thread James Coleman
On Wed, Aug 19, 2020 at 3:16 AM Heikki Linnakangas wrote: > > On 01/05/2020 05:20, James Coleman wrote: > > On Tue, Apr 28, 2020 at 8:25 AM Tomas Vondra > > wrote: > > ... > >> Any particular reasons to pick dynahash over simplehash? ISTM we're > >> using simplehash elsewhere in the executor (gro

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-08-19 Thread Heikki Linnakangas
On 01/05/2020 05:20, James Coleman wrote: On Tue, Apr 28, 2020 at 8:25 AM Tomas Vondra wrote: ... Any particular reasons to pick dynahash over simplehash? ISTM we're using simplehash elsewhere in the executor (grouping, tidbitmap, ...), while there are not many places using dynahash for simple

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-30 Thread James Coleman
On Tue, Apr 28, 2020 at 8:25 AM Tomas Vondra wrote: ... > Any particular reasons to pick dynahash over simplehash? ISTM we're > using simplehash elsewhere in the executor (grouping, tidbitmap, ...), > while there are not many places using dynahash for simple short-lived > hash tables. Of course, t

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-29 Thread Tomas Vondra
On Wed, Apr 29, 2020 at 11:34:24AM -0400, James Coleman wrote: On Wed, Apr 29, 2020 at 11:17 AM Tomas Vondra wrote: On Wed, Apr 29, 2020 at 10:26:12AM -0400, James Coleman wrote: >On Tue, Apr 28, 2020 at 7:05 PM Andres Freund wrote: >> >> Hi, >> >> On 2020-04-28 18:22:20 -0400, James Coleman

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-29 Thread James Coleman
On Wed, Apr 29, 2020 at 11:17 AM Tomas Vondra wrote: > > On Wed, Apr 29, 2020 at 10:26:12AM -0400, James Coleman wrote: > >On Tue, Apr 28, 2020 at 7:05 PM Andres Freund wrote: > >> > >> Hi, > >> > >> On 2020-04-28 18:22:20 -0400, James Coleman wrote: > >> > I cc'd Andres given his commit introduc

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-29 Thread Tomas Vondra
On Wed, Apr 29, 2020 at 10:26:12AM -0400, James Coleman wrote: On Tue, Apr 28, 2020 at 7:05 PM Andres Freund wrote: Hi, On 2020-04-28 18:22:20 -0400, James Coleman wrote: > I cc'd Andres given his commit introduced simplehash, so I figured > he'd probably have a few pointers on when each one

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-29 Thread James Coleman
On Tue, Apr 28, 2020 at 7:05 PM Andres Freund wrote: > > Hi, > > On 2020-04-28 18:22:20 -0400, James Coleman wrote: > > I cc'd Andres given his commit introduced simplehash, so I figured > > he'd probably have a few pointers on when each one might be useful. > > [...] > > Do you have any thoughts

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Andres Freund
Hi, On 2020-04-28 18:22:20 -0400, James Coleman wrote: > I cc'd Andres given his commit introduced simplehash, so I figured > he'd probably have a few pointers on when each one might be useful. > [...] > Do you have any thoughts on what the trade-offs/use-cases etc. are for > dynahash versus simpl

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Tomas Vondra
On Tue, Apr 28, 2020 at 06:22:20PM -0400, James Coleman wrote: I cc'd Andres given his commit introduced simplehash, so I figured he'd probably have a few pointers on when each one might be useful. On Tue, Apr 28, 2020 at 8:39 AM James Coleman wrote: ... > Any particular reasons to pick dynaha

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
I cc'd Andres given his commit introduced simplehash, so I figured he'd probably have a few pointers on when each one might be useful. On Tue, Apr 28, 2020 at 8:39 AM James Coleman wrote: ... > > Any particular reasons to pick dynahash over simplehash? ISTM we're > > using simplehash elsewhere in

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
On Tue, Apr 28, 2020 at 1:40 PM Pavel Stehule wrote: > > > > út 28. 4. 2020 v 18:17 odesílatel James Coleman napsal: >> >> On Tue, Apr 28, 2020 at 11:18 AM Pavel Stehule >> wrote: >> > >> > >> > >> > út 28. 4. 2020 v 16:48 odesílatel Tomas Vondra >> > napsal: >> >> >> >> On Tue, Apr 28, 2020

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Pavel Stehule
út 28. 4. 2020 v 18:17 odesílatel James Coleman napsal: > On Tue, Apr 28, 2020 at 11:18 AM Pavel Stehule > wrote: > > > > > > > > út 28. 4. 2020 v 16:48 odesílatel Tomas Vondra < > tomas.von...@2ndquadrant.com> napsal: > >> > >> On Tue, Apr 28, 2020 at 03:43:43PM +0200, Pavel Stehule wrote: > >>

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
On Tue, Apr 28, 2020 at 11:18 AM Pavel Stehule wrote: > > > > út 28. 4. 2020 v 16:48 odesílatel Tomas Vondra > napsal: >> >> On Tue, Apr 28, 2020 at 03:43:43PM +0200, Pavel Stehule wrote: >> >út 28. 4. 2020 v 15:26 odesílatel Tomas Vondra >> > >> >napsal: >> > >> >> ... >> >> >> >> >I'm not so

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Pavel Stehule
út 28. 4. 2020 v 16:48 odesílatel Tomas Vondra napsal: > On Tue, Apr 28, 2020 at 03:43:43PM +0200, Pavel Stehule wrote: > >út 28. 4. 2020 v 15:26 odesílatel Tomas Vondra < > tomas.von...@2ndquadrant.com> > >napsal: > > > >> ... > >> > >> >I'm not so concerned about this in any query where we have

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Tomas Vondra
On Tue, Apr 28, 2020 at 03:43:43PM +0200, Pavel Stehule wrote: út 28. 4. 2020 v 15:26 odesílatel Tomas Vondra napsal: ... >I'm not so concerned about this in any query where we have a real FROM >clause because even if we end up with only one row, the relative >penalty is low, and the potentia

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Pavel Stehule
út 28. 4. 2020 v 15:26 odesílatel Tomas Vondra napsal: > On Tue, Apr 28, 2020 at 08:39:18AM -0400, James Coleman wrote: > >On Tue, Apr 28, 2020 at 8:25 AM Tomas Vondra > > wrote: > >> > >> On Mon, Apr 27, 2020 at 09:04:09PM -0400, James Coleman wrote: > >> >On Sun, Apr 26, 2020 at 7:41 PM James C

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Tomas Vondra
On Tue, Apr 28, 2020 at 08:39:18AM -0400, James Coleman wrote: On Tue, Apr 28, 2020 at 8:25 AM Tomas Vondra wrote: On Mon, Apr 27, 2020 at 09:04:09PM -0400, James Coleman wrote: >On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: >> >> On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra >> wrote:

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
On Tue, Apr 28, 2020 at 8:25 AM Tomas Vondra wrote: > > On Mon, Apr 27, 2020 at 09:04:09PM -0400, James Coleman wrote: > >On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: > >> > >> On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra > >> wrote: > >> > > >> > On Sun, Apr 26, 2020 at 02:46:19PM -0400

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread Tomas Vondra
On Mon, Apr 27, 2020 at 09:04:09PM -0400, James Coleman wrote: On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra wrote: > > On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: > >On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra > > wrote:

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-27 Thread James Coleman
On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: > > On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra > wrote: > > > > On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: > > >On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra > > > wrote: > > >> > > >> On Sat, Apr 25, 2020 at 06:47:41PM -04

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-27 Thread James Coleman
On Sun, Apr 26, 2020 at 11:44 PM David Rowley wrote: > > On Mon, 27 Apr 2020 at 15:12, James Coleman wrote: > > While working on this I noticed that dynahash.c line 499 has this assertion: > > > > Assert(info->entrysize >= info->keysize); > > > > Do you by any chance know why the entry would need

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread David Rowley
On Mon, 27 Apr 2020 at 15:12, James Coleman wrote: > While working on this I noticed that dynahash.c line 499 has this assertion: > > Assert(info->entrysize >= info->keysize); > > Do you by any chance know why the entry would need to be larger than the key? Larger or equal. They'd be equal if you

Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread James Coleman
On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: > > On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra > wrote: > > > > On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: > > >On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra > > > wrote: > > >> > > >> On Sat, Apr 25, 2020 at 06:47:41PM -04

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread James Coleman
On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra wrote: > > On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: > >On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra > > wrote: > >> > >> On Sat, Apr 25, 2020 at 06:47:41PM -0400, James Coleman wrote: > >> >On Sat, Apr 25, 2020 at 5:41 PM David Row

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread Tomas Vondra
On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra wrote: On Sat, Apr 25, 2020 at 06:47:41PM -0400, James Coleman wrote: >On Sat, Apr 25, 2020 at 5:41 PM David Rowley wrote: >> >> On Sun, 26 Apr 2020 at 00:40, Tomas Vondra wrote: >> >

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread James Coleman
On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra wrote: > > On Sat, Apr 25, 2020 at 06:47:41PM -0400, James Coleman wrote: > >On Sat, Apr 25, 2020 at 5:41 PM David Rowley wrote: > >> > >> On Sun, 26 Apr 2020 at 00:40, Tomas Vondra > >> wrote: > >> > This reminds me our attempts to add bloom filters

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-25 Thread Tomas Vondra
On Sat, Apr 25, 2020 at 06:47:41PM -0400, James Coleman wrote: On Sat, Apr 25, 2020 at 5:41 PM David Rowley wrote: On Sun, 26 Apr 2020 at 00:40, Tomas Vondra wrote: > This reminds me our attempts to add bloom filters to hash joins, which I > think ran into mostly the same challenge of decidin

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-25 Thread James Coleman
On Sat, Apr 25, 2020 at 5:41 PM David Rowley wrote: > > On Sun, 26 Apr 2020 at 00:40, Tomas Vondra > wrote: > > This reminds me our attempts to add bloom filters to hash joins, which I > > think ran into mostly the same challenge of deciding when the bloom > > filter can be useful and is worth t

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-25 Thread David Rowley
On Sun, 26 Apr 2020 at 00:40, Tomas Vondra wrote: > This reminds me our attempts to add bloom filters to hash joins, which I > think ran into mostly the same challenge of deciding when the bloom > filter can be useful and is worth the extra work. Speaking of that, it would be interesting to see h

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-25 Thread Tomas Vondra
On Fri, Apr 24, 2020 at 09:22:34PM -0400, James Coleman wrote: On Fri, Apr 24, 2020 at 5:55 PM Tomas Vondra wrote: On Fri, Apr 24, 2020 at 09:38:54AM -0400, James Coleman wrote: >On Thu, Apr 23, 2020 at 10:55 AM Tomas Vondra > wrote: >> >> On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Colema

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-24 Thread James Coleman
On Fri, Apr 24, 2020 at 5:55 PM Tomas Vondra wrote: > > On Fri, Apr 24, 2020 at 09:38:54AM -0400, James Coleman wrote: > >On Thu, Apr 23, 2020 at 10:55 AM Tomas Vondra > > wrote: > >> > >> On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Coleman wrote: > >> >On Thu, Apr 23, 2020 at 8:47 AM Tomas Vo

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-24 Thread Tomas Vondra
On Fri, Apr 24, 2020 at 09:38:54AM -0400, James Coleman wrote: On Thu, Apr 23, 2020 at 10:55 AM Tomas Vondra wrote: On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Coleman wrote: >On Thu, Apr 23, 2020 at 8:47 AM Tomas Vondra > wrote: >> >> On Mon, Apr 20, 2020 at 09:27:34PM -0400, James Colema

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-24 Thread James Coleman
On Thu, Apr 23, 2020 at 10:55 AM Tomas Vondra wrote: > > On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Coleman wrote: > >On Thu, Apr 23, 2020 at 8:47 AM Tomas Vondra > > wrote: > >> > >> On Mon, Apr 20, 2020 at 09:27:34PM -0400, James Coleman wrote: > >> >Over in "execExprInterp() questions / Ho

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-23 Thread Andres Freund
Hi, On 2020-04-24 10:09:36 +1200, David Rowley wrote: > If single comparison for a binary search costs about the same as an > equality check, then wouldn't the crossover point be much lower than > 64? The costs aren't quite as simple as that though. Binary search usually has issues with cache mis

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-23 Thread David Rowley
On Fri, 24 Apr 2020 at 02:56, Tomas Vondra wrote: > > On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Coleman wrote: > >On Thu, Apr 23, 2020 at 8:47 AM Tomas Vondra > > wrote: > >As to the choice of 9 elements: I just picked that as a starting > >point; Andres had previously commented off hand tha

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-23 Thread Tomas Vondra
On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Coleman wrote: On Thu, Apr 23, 2020 at 8:47 AM Tomas Vondra wrote: On Mon, Apr 20, 2020 at 09:27:34PM -0400, James Coleman wrote: >Over in "execExprInterp() questions / How to improve scalar array op >expr eval?" [1] I'd mused about how we might

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-23 Thread James Coleman
On Thu, Apr 23, 2020 at 8:47 AM Tomas Vondra wrote: > > On Mon, Apr 20, 2020 at 09:27:34PM -0400, James Coleman wrote: > >Over in "execExprInterp() questions / How to improve scalar array op > >expr eval?" [1] I'd mused about how we might be able to optimized > >scalar array ops with OR'd semantic

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-23 Thread Tomas Vondra
On Mon, Apr 20, 2020 at 09:27:34PM -0400, James Coleman wrote: Over in "execExprInterp() questions / How to improve scalar array op expr eval?" [1] I'd mused about how we might be able to optimized scalar array ops with OR'd semantics. This patch implements a binary search for such expressions w

Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-20 Thread James Coleman
Over in "execExprInterp() questions / How to improve scalar array op expr eval?" [1] I'd mused about how we might be able to optimized scalar array ops with OR'd semantics. This patch implements a binary search for such expressions when the array argument is a constant so that we can avoid needing