Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-07 Thread James Coleman
On Tue, Apr 7, 2020 at 12:25 AM Justin Pryzby wrote: > > On Mon, Apr 06, 2020 at 09:57:22PM +0200, Tomas Vondra wrote: > > I've pushed the fist part of this patch series - I've reorganized it a > > I scanned through this again post-commit. Find attached some suggestions. > > Shouldn't non-text ex

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-07 Thread James Coleman
On Tue, Apr 7, 2020 at 7:02 PM Tomas Vondra wrote: > > On Mon, Apr 06, 2020 at 11:25:21PM -0500, Justin Pryzby wrote: > >On Mon, Apr 06, 2020 at 09:57:22PM +0200, Tomas Vondra wrote: > >> I've pushed the fist part of this patch series - I've reorganized it a > > > >I scanned through this again pos

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-07 Thread James Coleman
On Tue, Apr 7, 2020 at 7:58 PM Tomas Vondra wrote: > > On Tue, Apr 07, 2020 at 07:50:26PM -0400, James Coleman wrote: > >On Tue, Apr 7, 2020 at 7:02 PM Tomas Vondra > > wrote: > >> > >> On Mon, Apr 06, 2020 at 11:25:21PM -0500, Justin Pryzby wrote: > >&

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-08 Thread James Coleman
On Wed, Apr 8, 2020 at 9:43 AM Tomas Vondra wrote: > > On Wed, Apr 08, 2020 at 12:51:05PM +0200, Tomas Vondra wrote: > >On Tue, Apr 07, 2020 at 11:54:23PM -0400, Tom Lane wrote: > >>hyrax is not too happy with this test: > >> > >>https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax&dt=202

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-08 Thread James Coleman
On Wed, Apr 8, 2020 at 11:02 AM Tomas Vondra wrote: > > On Wed, Apr 08, 2020 at 04:08:39PM +0200, Tomas Vondra wrote: > >On Wed, Apr 08, 2020 at 09:54:42AM -0400, James Coleman wrote: > >>On Wed, Apr 8, 2020 at 9:43 AM Tomas Vondra > >> wrote: > >>> >

Re: [PATCH] Incremental sort

2020-04-08 Thread James Coleman
On Wed, Apr 8, 2020 at 11:29 AM David Steele wrote: > > On 4/8/20 11:13 AM, James Coleman wrote: > >> > >> James, can you verify it that's still true? > > I marked this entry as committed in the 2020-03 CF but it's not clear to > me if that's enti

Re: Multiple FPI_FOR_HINT for the same block during killing btree index items

2020-04-09 Thread James Coleman
On Thu, Apr 9, 2020 at 3:05 PM Peter Geoghegan wrote: > > On Wed, Apr 8, 2020 at 10:56 PM Masahiko Sawada > wrote: > > Here is the reproducer: > > What version of Postgres did you notice the actual customer issue on? > I ask because I wonder if the work on B-Tree indexes in Postgres 12 > affects

Re: Multiple FPI_FOR_HINT for the same block during killing btree index items

2020-04-09 Thread James Coleman
On Thu, Apr 9, 2020 at 8:32 PM Peter Geoghegan wrote: > > On Thu, Apr 9, 2020 at 5:25 PM Peter Geoghegan wrote: > > Was this a low cardinality index in the way I describe? If it was, > > then we can hope (and maybe even verify) that the Postgres 12 work > > noticeably ameliorates the problem. > >

Re: Multiple FPI_FOR_HINT for the same block during killing btree index items

2020-04-10 Thread James Coleman
On Thu, Apr 9, 2020 at 10:08 PM Peter Geoghegan wrote: > > On Thu, Apr 9, 2020 at 6:47 PM James Coleman wrote: > > I believe the write pattern to this table likely looks like: > > - INSERT > > - UPDATE > > - DELETE > > for every row. But tomorrow I can do

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-10 Thread James Coleman
One thing I just noticed and had a question about: in preparePresortedCols (which sets up a function call context), do we need to call pg_proc_aclcheck? James

execExprInterp() questions / How to improve scalar array op expr eval?

2020-04-11 Thread James Coleman
Short version: In what I'm currently working on I had a few questions about arrays and the execExpr/execExprInterp framework that didn't seem obviously answered in the code or README. - Does the execExpr/execExprInterp framework allow a scalar array op to get an already expanded array (unless I'm

Re: execExprInterp() questions / How to improve scalar array op expr eval?

2020-04-11 Thread James Coleman
On Sat, Apr 11, 2020 at 2:01 PM Andres Freund wrote: > > Hi, > > > Tom, CCing you because of expanded datum question at bottom. > > > On 2020-04-11 08:58:46 -0400, James Coleman wrote: > > - Does the execExpr/execExprInterp framework allow a scalar array op > &

Re: execExprInterp() questions / How to improve scalar array op expr eval?

2020-04-11 Thread James Coleman
On Sat, Apr 11, 2020 at 3:33 PM Tom Lane wrote: > > Andres Freund writes: > > On 2020-04-11 08:58:46 -0400, James Coleman wrote: > >> - Does the execExpr/execExprInterp framework allow a scalar array op > >> to get an already expanded array (unless I'm mis

Re: execExprInterp() questions / How to improve scalar array op expr eval?

2020-04-11 Thread James Coleman
On Sat, Apr 11, 2020 at 3:57 PM James Coleman wrote: > .. > > It seems like this might be somewhat related to the currently-moribund > > patch to allow caching of the values of stable subexpressions from > > one execution to the next. If we had that infrastructure you could

Re: execExprInterp() questions / How to improve scalar array op expr eval?

2020-04-12 Thread James Coleman
On Sat, Apr 11, 2020 at 5:32 PM Andres Freund wrote: > > Hi, > > On 2020-04-11 15:53:11 -0400, James Coleman wrote: > > On Sat, Apr 11, 2020 at 2:01 PM Andres Freund wrote: > > > > - If not, is there a way in that framework to know if the array expr > > >

Re: execExprInterp() questions / How to improve scalar array op expr eval?

2020-04-13 Thread James Coleman
I've read through all of the previous discussions related to stable subexpression caching, and I'm planning to send a summary email with all of those links in one place. But I also happened to stumble upon mention in the TODO of some email discussion way back in 2007 where Tom suggested [1] we sho

Re: sqlsmith crash incremental sort

2020-04-14 Thread James Coleman
On Sun, Apr 12, 2020 at 8:09 PM Tomas Vondra wrote: > > On Sun, Apr 12, 2020 at 12:44:45AM +0200, Tomas Vondra wrote: > >Hi, > > > >I've looked into this a bit, and at first I thought that maybe the > >issue is in how cost_incremental_sort picks the EC members. It simply > >does this: > > > >E

Re: [DOC] Document concurrent index builds waiting on each other

2020-04-15 Thread James Coleman
On Wed, Mar 25, 2020 at 3:58 PM Andres Freund wrote: > > Hi, > > On 2020-03-25 16:30:10 -0300, Alvaro Herrera wrote: > > I posted this in November > > https://postgr.es/m/20191101203310.GA12239@alvherre.pgsql but I didn't > > put time to go through the issues there. > > Oh, missed that. > > > > I

Re: Incremental sorts and EXEC_FLAG_REWIND

2020-04-15 Thread James Coleman
On Tue, Apr 14, 2020 at 2:53 AM Michael Paquier wrote: > > Hi, > > When initializing an incremental sort node, we have the following as > of ExecInitIncrementalSort(): > /* > * Incremental sort can't be used with either EXEC_FLAG_REWIND, > * EXEC_FLAG_BACKWARD or EXEC_FLAG_MARK, beca

Re: sqlsmith crash incremental sort

2020-04-15 Thread James Coleman
On Wed, Apr 15, 2020 at 10:47 AM Tomas Vondra wrote: > > On Tue, Apr 14, 2020 at 01:16:33PM -0400, James Coleman wrote: > >On Sun, Apr 12, 2020 at 8:09 PM Tomas Vondra > > wrote: > >> > >> On Sun, Apr 12, 2020 at 12:44:45AM +0200, Tomas Vondra wrote: > >&

Re: Incremental sorts and EXEC_FLAG_REWIND

2020-04-15 Thread James Coleman
On Wed, Apr 15, 2020 at 11:02 AM James Coleman wrote: > > On Tue, Apr 14, 2020 at 2:53 AM Michael Paquier wrote: > > > > Hi, > > > > When initializing an incremental sort node, we have the following as > > of ExecInitIncrementalSort(): > > /* >

Re: [DOC] Document concurrent index builds waiting on each other

2020-04-15 Thread James Coleman
On Wed, Apr 15, 2020 at 6:31 PM Andres Freund wrote: > > Hi, > > On 2020-04-15 09:31:58 -0400, James Coleman wrote: > > On Wed, Mar 25, 2020 at 3:58 PM Andres Freund wrote: > > > On 2020-03-25 16:30:10 -0300, Alvaro Herrera wrote: > > > > I posted this

Re: sqlsmith crash incremental sort

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 8:22 AM Richard Guo wrote: > > > On Thu, Apr 16, 2020 at 6:35 PM Richard Guo wrote: >> >> >> On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra >> wrote: >>> >>> >>> Well, yeah. The problem is the Unique simply compares the columns in the >>> order it sees them, and it does n

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-16 Thread James Coleman
On Fri, Apr 10, 2020 at 10:12 AM James Coleman wrote: > > One thing I just noticed and had a question about: in > preparePresortedCols (which sets up a function call context), do we > need to call pg_proc_aclcheck? Background: this came up because I noticed that pg_proc_aclcheck is c

Re: sqlsmith crash incremental sort

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 8:54 PM Tomas Vondra wrote: > > On Wed, Apr 15, 2020 at 11:26:12AM -0400, James Coleman wrote: > >On Wed, Apr 15, 2020 at 10:47 AM Tomas Vondra > > wrote: > >> > >> ... > >> > >> Yeah. And I'm not even sure havin

Re: [DOC] Document concurrent index builds waiting on each other

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 6:12 PM Andres Freund wrote: > > Hi, > > On 2020-04-15 21:44:48 -0400, James Coleman wrote: > > On Wed, Apr 15, 2020 at 6:31 PM Andres Freund wrote: > > > If it's about the xmin horizon for vacuum: I think we could probably > > >

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-16 Thread James Coleman
On Thu, Apr 16, 2020 at 1:10 PM Tom Lane wrote: > > James Coleman writes: > > On Fri, Apr 10, 2020 at 10:12 AM James Coleman wrote: > >> One thing I just noticed and had a question about: in > >> preparePresortedCols (which sets up a function call con

Re: execExprInterp() questions / How to improve scalar array op expr eval?

2020-04-17 Thread James Coleman
On Mon, Apr 13, 2020 at 10:40 AM James Coleman wrote: > > I've read through all of the previous discussions related to stable > subexpression caching, and I'm planning to send a summary email with > all of those links in one place. > > But I also happened to stumble

Summary: State of Caching Stable Subexpressions

2020-04-17 Thread James Coleman
I've recently been thinking about some optimizations to scalar array op expression evaluation [1], and Tom mentioned that I might want to consider looking into previous efforts into caching stable subexpressions as a component of that (particularly if I wanted it to be useful for more than constant

Re: sqlsmith crash incremental sort

2020-04-18 Thread James Coleman
On Thu, Apr 16, 2020 at 9:26 PM Tom Lane wrote: > > Tomas Vondra writes: > > I think we have essentially three options: > > 1) assuming there's just a single group > > 2) assuming each row is a separate group > > 3) something in between > > If (1) and (2) are worst/best-case scenarios, maybe we s

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-19 Thread James Coleman
On Sat, Apr 18, 2020 at 10:36 PM Justin Pryzby wrote: > > On Tue, Apr 07, 2020 at 10:53:05AM -0500, Justin Pryzby wrote: > > On Tue, Apr 07, 2020 at 08:40:30AM -0400, James Coleman wrote: > > > > And, should it use two spaces before "Sort Method", "Memory&qu

Re: Incremental sorts and EXEC_FLAG_REWIND

2020-04-19 Thread James Coleman
On Wed, Apr 15, 2020 at 2:04 PM James Coleman wrote: > > On Wed, Apr 15, 2020 at 11:02 AM James Coleman wrote: > > > > On Tue, Apr 14, 2020 at 2:53 AM Michael Paquier wrote: > > > > > > Hi, > > > > > > When initializing an in

Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-20 Thread James Coleman
Over in "execExprInterp() questions / How to improve scalar array op expr eval?" [1] I'd mused about how we might be able to optimized scalar array ops with OR'd semantics. This patch implements a binary search for such expressions when the array argument is a constant so that we can avoid needing

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-23 Thread James Coleman
On Thu, Apr 23, 2020 at 8:47 AM Tomas Vondra wrote: > > On Mon, Apr 20, 2020 at 09:27:34PM -0400, James Coleman wrote: > >Over in "execExprInterp() questions / How to improve scalar array op > >expr eval?" [1] I'd mused about how we might be able to optimi

Re: [PATCH] Fix division by zero (explain.c)

2020-04-23 Thread James Coleman
On Thu, Apr 23, 2020 at 8:38 AM Ranier Vilela wrote: > > Hi, > > Per Coverity. > > If has 0 full groups, "we don't need to do anything" and need goes to next. > Otherwise a integer division by zero, can raise. > > comments extracted trom explain.c: > /* > * Since we never have any prefix groups u

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-24 Thread James Coleman
On Thu, Apr 23, 2020 at 10:55 AM Tomas Vondra wrote: > > On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Coleman wrote: > >On Thu, Apr 23, 2020 at 8:47 AM Tomas Vondra > > wrote: > >> > >> On Mon, Apr 20, 2020 at 09:27:34PM -0400, James Coleman wrote: >

Re: Incremental sorts and EXEC_FLAG_REWIND

2020-04-24 Thread James Coleman
On Sun, Apr 19, 2020 at 12:14 PM James Coleman wrote: > > On Wed, Apr 15, 2020 at 2:04 PM James Coleman wrote: > > > > On Wed, Apr 15, 2020 at 11:02 AM James Coleman wrote: > > > > > > On Tue, Apr 14, 2020 at 2:53 AM Michael Paquier > > > wr

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-24 Thread James Coleman
On Fri, Apr 24, 2020 at 5:55 PM Tomas Vondra wrote: > > On Fri, Apr 24, 2020 at 09:38:54AM -0400, James Coleman wrote: > >On Thu, Apr 23, 2020 at 10:55 AM Tomas Vondra > > wrote: > >> > >> On Thu, Apr 23, 2020 at 09:02:26AM -0400, James Coleman wrote: > &g

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-25 Thread James Coleman
On Sat, Apr 25, 2020 at 5:41 PM David Rowley wrote: > > On Sun, 26 Apr 2020 at 00:40, Tomas Vondra > wrote: > > This reminds me our attempts to add bloom filters to hash joins, which I > > think ran into mostly the same challenge of deciding when the bloom > > filter can be useful and is worth t

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread James Coleman
On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra wrote: > > On Sat, Apr 25, 2020 at 06:47:41PM -0400, James Coleman wrote: > >On Sat, Apr 25, 2020 at 5:41 PM David Rowley wrote: > >> > >> On Sun, 26 Apr 2020 at 00:40, Tomas Vondra > >> wrote: > >

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread James Coleman
On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra wrote: > > On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: > >On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra > > wrote: > >> > >> On Sat, Apr 25, 2020 at 06:47:41PM -0400, James Coleman wrote: > &g

Re: doc review for v13

2020-04-26 Thread James Coleman
On Sun, Apr 26, 2020 at 12:13 PM Justin Pryzby wrote: > > On Tue, Apr 14, 2020 at 02:47:54PM +0900, Michael Paquier wrote: > > On Sun, Apr 12, 2020 at 04:35:45PM -0500, Justin Pryzby wrote: > > > Added a few more. > > > And rebased on top of dbc60c5593f26dc777a3be032bff4fb4eab1ddd1 > > > > Thanks

Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-26 Thread James Coleman
On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: > > On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra > wrote: > > > > On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: > > >On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra > > > wrote: > >

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-27 Thread James Coleman
On Sun, Apr 26, 2020 at 11:44 PM David Rowley wrote: > > On Mon, 27 Apr 2020 at 15:12, James Coleman wrote: > > While working on this I noticed that dynahash.c line 499 has this assertion: > > > > Assert(info->entrysize >= info->keysize); > > > >

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-27 Thread James Coleman
On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: > > On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra > wrote: > > > > On Sun, Apr 26, 2020 at 02:46:19PM -0400, James Coleman wrote: > > >On Sat, Apr 25, 2020 at 8:31 PM Tomas Vondra > > > wrote: > >

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
On Tue, Apr 28, 2020 at 8:25 AM Tomas Vondra wrote: > > On Mon, Apr 27, 2020 at 09:04:09PM -0400, James Coleman wrote: > >On Sun, Apr 26, 2020 at 7:41 PM James Coleman wrote: > >> > >> On Sun, Apr 26, 2020 at 4:49 PM Tomas Vondra > >> wrote: > >

Re: pg_rewind docs correction

2020-04-28 Thread James Coleman
On Tue, Apr 28, 2020 at 12:31 AM Michael Paquier wrote: > > On Mon, Mar 09, 2020 at 09:26:17AM -0400, James Coleman wrote: > >> - pg_stat_tmp/, and > >> - pg_subtrans/ are omitted from the data copied > >> - from the source cluster. Any

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
On Tue, Apr 28, 2020 at 11:18 AM Pavel Stehule wrote: > > > > út 28. 4. 2020 v 16:48 odesílatel Tomas Vondra > napsal: >> >> On Tue, Apr 28, 2020 at 03:43:43PM +0200, Pavel Stehule wrote: >> >út 28. 4. 2020 v 15:26 odesílatel Tomas Vondra >> > >> >napsal: >> > >> >> ... >> >> >> >> >I'm not so

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
On Tue, Apr 28, 2020 at 1:40 PM Pavel Stehule wrote: > > > > út 28. 4. 2020 v 18:17 odesílatel James Coleman napsal: >> >> On Tue, Apr 28, 2020 at 11:18 AM Pavel Stehule >> wrote: >> > >> > >> > >> > út 28. 4. 2020 v 16:48 odesí

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-28 Thread James Coleman
I cc'd Andres given his commit introduced simplehash, so I figured he'd probably have a few pointers on when each one might be useful. On Tue, Apr 28, 2020 at 8:39 AM James Coleman wrote: ... > > Any particular reasons to pick dynahash over simplehash? ISTM we're > >

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-29 Thread James Coleman
On Tue, Apr 28, 2020 at 7:05 PM Andres Freund wrote: > > Hi, > > On 2020-04-28 18:22:20 -0400, James Coleman wrote: > > I cc'd Andres given his commit introduced simplehash, so I figured > > he'd probably have a few pointers on when each one might be useful. >

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-29 Thread James Coleman
On Wed, Apr 29, 2020 at 11:17 AM Tomas Vondra wrote: > > On Wed, Apr 29, 2020 at 10:26:12AM -0400, James Coleman wrote: > >On Tue, Apr 28, 2020 at 7:05 PM Andres Freund wrote: > >> > >> Hi, > >> > >> On 2020-04-28 18:22:20 -0400, James Cole

Comment simplehash/dynahash trade-offs

2020-04-30 Thread James Coleman
70b30194efb4b7d0e622afc3085ecd49b6 Mon Sep 17 00:00:00 2001 From: James Coleman Date: Thu, 30 Apr 2020 21:39:04 -0400 Subject: [PATCH v1 2/3] Improve simplehash usage notes --- src/include/lib/simplehash.h | 13 + 1 file changed, 13 insertions(+) diff --git a/src/include/lib/simp

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2020-04-30 Thread James Coleman
01df064f96e5f2a3be2b1c736bacc485 Mon Sep 17 00:00:00 2001 From: James Coleman Date: Tue, 28 Apr 2020 21:33:12 -0400 Subject: [PATCH v3 3/3] Try simple hash --- src/backend/executor/execExpr.c | 24 +++ src/backend/executor/execExprInterp.c | 99 +-- src/includ

Re: pg_rewind docs correction

2020-05-01 Thread James Coleman
On Fri, May 1, 2020 at 4:46 AM Michael Paquier wrote: > > On Wed, Apr 29, 2020 at 09:15:06AM +0900, Michael Paquier wrote: > > I am letting that aside for a couple of days to see if others have > > more comments, and will likely commit it after an extra lookup. > > And applied after an extra looku

Re: Incremental sorts and EXEC_FLAG_REWIND

2020-05-07 Thread James Coleman
On Thu, May 7, 2020 at 2:57 PM Jonathan S. Katz wrote: > > On 4/24/20 6:57 PM, Tomas Vondra wrote: > > On Fri, Apr 24, 2020 at 04:35:02PM -0400, James Coleman wrote: > >> On Sun, Apr 19, 2020 at 12:14 PM James Coleman wrote: > >>> > >>> On Wed,

Re: [PATCH] Fix division by zero (explain.c)

2020-05-08 Thread James Coleman
On Fri, May 8, 2020 at 7:20 PM Tomas Vondra wrote: > > On Fri, May 08, 2020 at 07:25:36PM -0300, Ranier Vilela wrote: > >Em sex., 8 de mai. de 2020 às 19:02, Tomas Vondra < > >tomas.von...@2ndquadrant.com> escreveu: > > > >> On Thu, Apr 23, 2020 at 04:12:34

Re: Incremental sorts and EXEC_FLAG_REWIND

2020-05-08 Thread James Coleman
On Fri, May 8, 2020 at 7:14 PM Tomas Vondra wrote: > > On Fri, Apr 24, 2020 at 04:35:02PM -0400, James Coleman wrote: > >On Sun, Apr 19, 2020 at 12:14 PM James Coleman wrote: > >> > >> On Wed, Apr 15, 2020 at 2:04 PM James Coleman wrote: > >> > >

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-22 Thread James Coleman
bit in the attached patch to start). > I think the name clause_proved_for_null_test() is a bit weird, being in > the past tense. I'd maybe change "proved" to "proves". Changed. > s/exppresions/expresions/ in the test files. Fixed. James Coleman saop_is_not_null-v8.patch Description: Binary data

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-22 Thread James Coleman
false) but > false for B, so that definition doesn't hold. So I think the comment > is accurate, but I can reword if you have an idea of what you'd like > to see (I've tweaked a bit in the attached patch to start). I forgot to update in v8 so attaching v9. James Coleman saop_is_not_null-v9.patch Description: Binary data

Re: Index Skip Scan

2019-02-01 Thread James Coleman
hen skip scanning would be a dramatic improvement. I suppose that both requirements could be met by incorporating it into the existing index scanning code and also modifying to costing to (only when we have high confidence?) account for the optimization. I'm not sure if that makes things better than the current state of the patch or not. James Coleman

Re: 2019-03 CF Summary / Review - Tranche #2

2019-02-17 Thread James Coleman
"learn as you go", but since there's not a lot of information directly written on it I figured asking explicitly is the best way to learn the process better. Thanks, James Coleman

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-02-26 Thread James Coleman
WHAT, exactly?" --- and as these > examples show, being crystal clear on what it proves is essential. Given the new argument, I've reverted the name change. I also updated the tests with a new helper function "opaque_array" to make it easy to test cases where the planner can&#

printf ordering issues?

2018-12-11 Thread James Coleman
realized instead that the output from the executor is getting flushed on the *subsequent* query. Is there some kind of weird behavior that most hackers already know about and I'm missing? (Also I realize I can use elog(LOG...), but I'm still curious.) Thanks, James Coleman

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2018-12-17 Thread James Coleman
Is there anything I can do to solicit reviewers for the current CF? The patch is quite small and should be fairly simple to review. - James

Using Btree to Provide Sorting on Suffix Keys with LIMIT

2018-12-29 Thread James Coleman
e worth showing yet, but I wanted to start the discussion on the design considerations while I continue work on the patch. - James Coleman

Re: Using Btree to Provide Sorting on Suffix Keys with LIMIT

2018-12-30 Thread James Coleman
On Sat, Dec 29, 2018 at 9:50 PM David Rowley wrote: > > On Sun, 30 Dec 2018 at 13:00, James Coleman wrote: > > Note that the index scan (or bitmap scan) nodes return all 1500 rows > > matching `bar_fk IN (1,2,3)`. After all rows are returned, that total > > set is ordere

Re: Using vim for developing porstres wiki article

2019-01-02 Thread James Coleman
a loss. I noticed this particularly since since just 2 or 3 days ago I myself had edited this section to add the softtabstop=0 option (the Vim default) so that if soft tabs are enabled in someone's general Vim config then hitting the tab key won't result in inserting 2 spaces while working in the Postgres source. Thanks, James Coleman

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-13 Thread James Coleman
e sad path test in addition to my original happy path test. Patch v3 attached. James Coleman saop_is_not_null-v3.patch Description: Binary data

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
uot;, but I figured it was worth getting this logically correct before attempting to optimize that especially since it already exists in one place. > Minor stylistic quibble: I don't like where you inserted the new code in > clause_is_strict_for, because it renders the comment a few lines abo

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
On Mon, Jan 14, 2019 at 11:08 AM Tom Lane wrote: > > James Coleman writes: > > On Sun, Jan 13, 2019 at 3:06 PM Tom Lane wrote: > >> There's still a logical problem here, which is that in order to > >> prove that the ScalarArrayOpExpr yields NULL when its LH

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
On Mon, Jan 14, 2019 at 11:34 AM Tom Lane wrote: > > James Coleman writes: > > On Mon, Jan 14, 2019 at 11:08 AM Tom Lane wrote: > >> I think these test cases don't actually prove much about the behavior > >> of your patch. Wouldn't they be handled by the

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
;ve also added the "parallel" case in predicate_refuted_by_simple_clause, but I haven't added a test for that yet. I also would like to add a test for the sad path to parallel the happy path computed array/cte test. While I add that though I wanted to get this updated version out to get feedback on the approach. James Coleman saop_is_not_null-v5.patch Description: Binary data

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-15 Thread James Coleman
ell as s_r_holds. I'd only expected the latter, since only "strongly_refuted_by = t". 3. The tests I have for refuting "(x + x) is null" show that both s_r_holds and w_r_holds. I'd expected these to be false. I'm attaching the current version of the patch with

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-15 Thread James Coleman
On Tue, Jan 15, 2019 at 3:53 PM Tom Lane wrote: > > James Coleman writes: > > [ saop_is_not_null-v6.patch ] > > I quite dislike taking the responsibility out of clause_is_strict_for > and putting it in the callers. Aside from requiring duplicative code, > it means

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-15 Thread James Coleman
, then it must imply the addition of itself is not null also? This is the root of the questions I had: James Coleman writes: > 1. The current code doesn't result in "strongly_implied_by = t" for > the "(x + x) is not null" case, but it does result in "s_i_hol

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-15 Thread James Coleman
On Tue, Jan 15, 2019 at 8:14 PM David Rowley wrote: > > On Wed, 16 Jan 2019 at 14:05, James Coleman wrote: > > At the risk of missing something obvious, I'm not sure I see a case > > where "x is not null" does not imply "(x + x) is not null", at le

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-16 Thread James Coleman
with array constants of > 100 values)? I dislike that it's not as obvious what's going on, but given that the code shouldn't care about array size anyway...so if there's an inclination to fewer tests that's the first place I'd look at cutting them. James Coleman

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-17 Thread James Coleman
On Wed, Jan 16, 2019 at 8:49 AM James Coleman wrote: > > On Tue, Jan 15, 2019 at 11:37 PM Tom Lane wrote: > > Well, as I said upthread, it seems like we need to think a bit more > > carefully about what it is that clause_is_strict_for is testing --- > > and if that e

Re: Using Btree to Provide Sorting on Suffix Keys with LIMIT

2019-01-18 Thread James Coleman
arantee ordering as well. Does that more real-world-ish example help place the usefulness of this? I think this goes beyond increasing the usefulness of indexes by requiring less specific indexes (incremental sort does this), but rather allows the index to support a kind of query you can't currently (as far as I'm aware) can't express in a performant way at call currently (other than a complex recursive cte or in some subset of cases a bunch of union statements -- one per array entry). James Coleman

Re: Index Skip Scan

2020-02-08 Thread James Coleman
On Sat, Feb 8, 2020 at 10:24 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Sat, Feb 08, 2020 at 03:22:17PM +0100, Tomas Vondra wrote: > > So in this case the skip scan is ~15x slower than the usual plan (index > > only scan + unique). The reason why this happens is pretty simple - to > >

Re: [DOC] Document concurrent index builds waiting on each other

2020-02-14 Thread James Coleman
I went ahead and registered this in the current only CF as https://commitfest.postgresql.org/27/2454/ Alvaro: Would you like to be added as a reviewer? James

Re: [DOC] Document concurrent index builds waiting on each other

2020-02-14 Thread James Coleman
On Sun, Sep 29, 2019 at 9:24 PM Michael Paquier wrote: > > On Sat, Sep 28, 2019 at 10:22:28PM -0300, Alvaro Herrera wrote: > > I always thought that create index concurrently was prevented from > > running concurrently in a table by the ShareUpdateExclusive lock that's > > held during the operatio

Re: [DOC] Document auto vacuum interruption

2020-02-14 Thread James Coleman
On Thu, Sep 19, 2019 at 5:34 AM Amit Kapila wrote: > > On Wed, Sep 18, 2019 at 10:25 AM Amit Kapila wrote: > > > > On Tue, Sep 17, 2019 at 5:48 PM James Coleman wrote: > > > > > > On Tue, Sep 17, 2019 at 2:21 AM Amit Kapila > > > wrote: > >

Re: [PATCH] Incremental sort

2020-03-04 Thread James Coleman
On Tue, Mar 3, 2020 at 1:43 PM Tomas Vondra wrote: > On Tue, Mar 03, 2020 at 12:17:22PM -0500, David Steele wrote: > >James and Tomas, > > > >On 1/21/20 10:03 AM, James Coleman wrote: > >>On Tue, Jan 21, 2020 at 9:58 AM Tomas Vondra > >> wrote: > >>

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-05 Thread James Coleman
On Tue, Jan 21, 2020 at 9:37 AM James Coleman wrote: > That being said, the patch also needs some more work on improving > EXPLAIN ANALYZE output (perhaps min/max/mean or median of > memory usage number of groups in each sort mode), and I think it's far > more feasible that

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-05 Thread James Coleman
On Thu, Mar 5, 2020 at 5:53 PM Tom Lane wrote: > James Coleman writes: > > I'm looking at this now, and realized that at least for parallel plans > the > > current patch tracks the tuplesort instrumentation whether or not an > > EXPLAIN ANALYZE is in process. >

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-03-07 Thread James Coleman
On Thu, Mar 5, 2020 at 6:45 PM Tom Lane wrote: > James Coleman writes: > > That's what I figured, but as I mentioned I've having trouble figuring > out > > how the fact that an analyze is in flight is determined. I assume it's > > something that live

Re: Allow to_date() and to_timestamp() to accept localized names

2020-03-07 Thread James Coleman
On Tue, Mar 3, 2020 at 11:09 AM Tom Lane wrote: > =?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?= < > juanjo.santama...@gmail.com> writes: > > On Sun, Mar 1, 2020 at 11:25 PM Tom Lane wrote: > >> Going once, going twice ... > > > You have moved this to better place, so none from me, and thank

Re: Allow to_date() and to_timestamp() to accept localized names

2020-03-07 Thread James Coleman
On Sat, Mar 7, 2020 at 9:31 PM Tom Lane wrote: > James Coleman writes: > > On master with a clean build (and configure re-run) and a fresh init-db, > > I'm seeing the collate.linux.utf8 test fail with the attached diff. > > -- to_char > SET lc_time TO 'tr

Re: Allow to_date() and to_timestamp() to accept localized names

2020-03-08 Thread James Coleman
On Sat, Mar 7, 2020 at 9:48 PM Tom Lane wrote: > James Coleman writes: > > On Sat, Mar 7, 2020 at 9:31 PM Tom Lane wrote: > >> Looks like you may not have Turkish locale installed? Try > >> locale -a | grep tr_TR > > > Hmm, when I grep the locales I see

Re: Allow to_date() and to_timestamp() to accept localized names

2020-03-08 Thread James Coleman
On Sun, Mar 8, 2020 at 7:17 AM Juan José Santamaría Flecha < juanjo.santama...@gmail.com> wrote: > > > On Sun, Mar 8, 2020 at 3:48 AM Tom Lane wrote: > >> James Coleman writes: >> > On Sat, Mar 7, 2020 at 9:31 PM Tom Lane wrote: >> >> Looks like yo

Re: Fastpath while arranging the changes in LSN order in logical decoding

2020-03-08 Thread James Coleman
On Saturday, March 7, 2020, Dilip Kumar wrote: > On Sat, Mar 7, 2020 at 9:59 AM Dilip Kumar wrote: > > > > On Sat, Mar 7, 2020 at 12:30 AM Andres Freund > wrote: > > > > > > Hi, > > > > > > On 2020-01-08 18:06:52 +0530, Dilip Kumar wrote: > > > > On Wed, 8 Jan 2020 at 5:28 PM, Heikki Linnakanga

Re: Allow to_date() and to_timestamp() to accept localized names

2020-03-08 Thread James Coleman
On Sun, Mar 8, 2020 at 10:42 AM James Coleman wrote: > On Sun, Mar 8, 2020 at 7:17 AM Juan José Santamaría Flecha < > juanjo.santama...@gmail.com> wrote: > >> >> >> On Sun, Mar 8, 2020 at 3:48 AM Tom Lane wrote: >> >>> James Coleman writes: >

Re: pg_rewind docs correction

2020-03-08 Thread James Coleman
On Tue, Sep 17, 2019 at 9:41 PM Michael Paquier wrote: > On Tue, Sep 17, 2019 at 08:38:18AM -0400, James Coleman wrote: > > I don't agree that that's a valid equivalency. I myself spent a lot of > > time trying to understand how this could possibly be true a while >

Re: pg_rewind docs correction

2020-03-08 Thread James Coleman
On Sun, Mar 8, 2020 at 5:13 PM James Coleman wrote: > > I realized I didn't previously add this to the CF; since it's not a new > patch I've added it to the current CF, but if this is incorrect please let > me know. > Hmm, looks like I can't add it to the

Re: Allow to_date() and to_timestamp() to accept localized names

2020-03-08 Thread James Coleman
On Sun, Mar 8, 2020 at 2:19 PM Tom Lane wrote: > I wrote: > > James Coleman writes: > >> I'm still interested in understanding why we're using the ISO locale > >> instead of the utf8 one in a utf8-labeled test though. > > > We are not. My und

Re: Allow to_date() and to_timestamp() to accept localized names

2020-03-08 Thread James Coleman
On Sun, Mar 8, 2020 at 6:03 PM Tom Lane wrote: > James Coleman writes: > > So just to confirm I understand, that implies that the issue is solely > that > > only the utf8 tr_TR set is installed by default on this machine, and the > > iso-8859-9 set is a hard requireme

Nicer error when connecting to standby with hot_standby=off

2020-03-08 Thread James Coleman
changing this value would result in a wire protocol change/something the client wants to know about? If so, I assume it's not reasonable to change the value, but would it still be reasonable to change the error text? Thanks, James Coleman

Re: pg_rewind docs correction

2020-03-09 Thread James Coleman
On Mon, Mar 9, 2020 at 2:59 AM Michael Paquier wrote: > On Sun, Mar 08, 2020 at 05:13:21PM -0400, James Coleman wrote: > > I've added the information about how the backup label control file is > > written, and updated the How It Works steps to refer to that separat

<    1   2   3   4   5   6   >