Re: Allow foreign keys to reference a superset of unique columns

2022-09-25 Thread James Coleman
ang Walther had commented similarly, but it appears that that idea got lost (or at least not responded to). I'd be happy to sign up to review an updated patch if you're interested in continuing this effort. If so, could you register the patch in the CF app (if not there already)? Thanks, James Coleman

Re: Allow foreign keys to reference a superset of unique columns

2022-09-26 Thread James Coleman
On Mon, Sep 26, 2022 at 2:28 AM Wolfgang Walther wrote: > > James Coleman: > > If we have a declared constraint on x,y where x is unique based on an > > index including on x I do not think we should have that fk constraint > > work differently than a constraint on x,y

Re: Consider parallel for lateral subqueries with limit

2022-09-26 Thread James Coleman
On Mon, Sep 26, 2022 at 10:37 AM Robert Haas wrote: > > On Thu, Sep 22, 2022 at 5:19 PM James Coleman wrote: > > > Your sample query gets a plan like this: > > > > > > Nested Loop Left Join (cost=0.00..1700245.00 rows=1 width=8) > > >-> S

Re: Allow foreign keys to reference a superset of unique columns

2022-09-26 Thread James Coleman
On Mon, Sep 26, 2022 at 9:59 AM Wolfgang Walther wrote: > > James Coleman: > > So the broader point I'm trying to make is that, as I understand it, > > indexes backing foreign key constraints is an implementation detail. > > The SQL standard details the behavi

Re: Allow foreign keys to reference a superset of unique columns

2022-09-26 Thread James Coleman
On Mon, Sep 26, 2022 at 10:04 AM Wolfgang Walther wrote: > > James Coleman: > > As I was reading through the email chain I had this thought: could you > > get the same benefit (or 90% of it anyway) by instead allowing the > > creation of a uniqueness constraint that co

Add hint about downloadable logs to CI README

2022-09-26 Thread James Coleman
I was wondering about how to debug failed builds on Cirrus CI, and after poking at the interface I realized we helpfully upload the logs from CI runs for user download. In an effort to save the next person a few minutes I thought the attached minor patch would help. Thanks, James Coleman v1

cirrus-ci cross-build interactions?

2022-09-26 Thread James Coleman
pushing the build [2] seems to be fine. I've double-checked there are no differences between the commits on the two builds (git diff shows no output). Is it possible we're missing some kind of necessary build isolation in the Cirrus CI scripting? Thanks, James Coleman 1: https://cirru

Re: cirrus-ci cross-build interactions?

2022-09-26 Thread James Coleman
On Mon, Sep 26, 2022 at 10:36 PM James Coleman wrote: > > I had a build on Cirrus CI fail tonight in what I have to assume was > either a problem with caching across builds or some such similar > flakiness. In the Debian task [1] I received this error: > > su postgres -c "

Re: Parallelize correlated subqueries that execute within each worker

2022-09-26 Thread James Coleman
On Mon, Mar 21, 2022 at 8:48 PM Andres Freund wrote: > > Hi, > > On 2022-01-22 20:25:19 -0500, James Coleman wrote: > > On the other hand this is a dramatically simpler patch series. > > Assuming the approach is sound, it should much easier to maintain than > > the

Re: cirrus-ci cross-build interactions?

2022-09-26 Thread James Coleman
On Mon, Sep 26, 2022 at 10:48 PM Andres Freund wrote: > > Hi, > > On 2022-09-26 22:36:24 -0400, James Coleman wrote: > > I had a build on Cirrus CI fail tonight in what I have to assume was > > either a problem with caching across builds or some such similar > > fl

Minor typo in generate_useful_gather_paths comment

2021-07-05 Thread James Coleman
While re-reading this code I found a small typo and fixed it (making the comment more explicit at the same time). Thanks, James v1-0001-Fix-typo-in-comment.patch Description: Binary data

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-05 Thread James Coleman
On Sat, Jun 12, 2021 at 11:07 AM David Rowley wrote: > > A few years ago I wrote a patch to implement the missing aggregate > combine functions for array_agg and string_agg [1]. In the end, the > patch was rejected due to some concern [2] that if we allow these > aggregates to run in parallel the

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-05 Thread James Coleman
On Mon, Jul 5, 2021 at 8:08 AM Ronan Dunklau wrote: > > > Ok, I reproduced that case, just not using a group by: by adding the group > > by a sort node is added in both cases (master and your patch), except that > > with your patch we sort on both keys and that doesn't really incur a > > performan

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread James Coleman
Adding David since this patch is likely a precondition for [1]. On Tue, Jul 6, 2021 at 2:15 AM Ronan Dunklau wrote: > > Hello, > > While testing the patch "Add proper planner support for ORDER BY / DISTINCT > aggregates" [0] I discovered the performance penalty from adding a sort node > essential

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-06 Thread James Coleman
On Tue, Jul 6, 2021 at 11:03 AM Ronan Dunklau wrote: > > Thank you for the review, I will address those shortly, but will answer some > questions in the meantime. > > > > First, the changes are lacking any explanatory comments. Probably we > > > should follow how nodeAgg does this and add both com

pg_rewind: warn when checkpoint hasn't happened after promotion

2022-06-04 Thread James Coleman
etects this condition and reports it as an error to the user. In the spirit of the new-ish "ensure shutdown" functionality I could imagine extending this to automatically issue a checkpoint when this situation is detected. I haven't started to code that up, however, wanting to first ge

Re: pg_rewind: warn when checkpoint hasn't happened after promotion

2022-06-06 Thread James Coleman
On Sat, Jun 4, 2022 at 9:39 AM Bharath Rupireddy wrote: > > On Sat, Jun 4, 2022 at 6:29 PM James Coleman wrote: > > > > A few weeks back I sent a bug report [1] directly to the -bugs mailing > > list, and I haven't seen any activity on it (maybe this is because I &g

Re: pg_rewind: warn when checkpoint hasn't happened after promotion

2022-06-06 Thread James Coleman
On Mon, Jun 6, 2022 at 1:26 AM Kyotaro Horiguchi wrote: > > At Sat, 4 Jun 2022 19:09:41 +0530, Bharath Rupireddy > wrote in > > On Sat, Jun 4, 2022 at 6:29 PM James Coleman wrote: > > > > > > A few weeks back I sent a bug report [1] directly to the -bugs maili

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-06-17 Thread James Coleman
here's maybe even some gains due to the smaller step > size. I didn't see that comment when working on this (it's quite a long unioned struct; I concur on adding an assert to catch it). This patch looks very reasonable to me though. James Coleman

Re: pg_rewind: warn when checkpoint hasn't happened after promotion

2022-07-05 Thread James Coleman
On Tue, Jul 5, 2022 at 2:39 PM Robert Haas wrote: > > On Sat, Jun 4, 2022 at 8:59 AM James Coleman wrote: > > A quick background refresher: after promoting a standby rewinding the > > former primary requires that a checkpoint have been completed on the > > new primary

Re: [Commitfest 2022-07] Patch Triage: Waiting on Author

2022-07-26 Thread James Coleman
m this CF, but I've been sufficiently busy that that hasn't happened. I'd like to just move these to the next CF. Thanks, James Coleman

Re: Parallelize correlated subqueries that execute within each worker

2021-09-07 Thread James Coleman
On Wed, Sep 1, 2021 at 7:06 AM Daniel Gustafsson wrote: > > > On 7 May 2021, at 18:30, James Coleman wrote: > > > ..here we are now, and I finally have this patch cleaned up > > enough to share. > > This patch no longer applies to HEAD, can you please submit a

Re: Parallelize correlated subqueries that execute within each worker

2021-09-08 Thread James Coleman
On Tue, Sep 7, 2021 at 11:06 AM Zhihong Yu wrote: > > > > On Tue, Sep 7, 2021 at 6:17 AM James Coleman wrote: >> >> On Wed, Sep 1, 2021 at 7:06 AM Daniel Gustafsson wrote: >> > >> > > On 7 May 2021, at 18:30, James Coleman wrote: >> > &

Document atthasmissing default optimization avoids verification table scan

2021-09-24 Thread James Coleman
null. That scan happens under an exclusive lock on the table, so it can have a meaningful impact on database "accessible uptime". I've attached a patch to document that the new mechanism also precludes that scan. Thanks, James Coleman v1-0001-Document-atthasmissing

Document spaces in .pgpass need to be escaped

2021-09-28 Thread James Coleman
A coworker has a space in a Postgres password and noticed .pgpass didn't work; escaping it fixed the issue. That requirement wasn't documented (despite other escaping requirements being documented), so I've attached a patch to add that comment. Thanks, James Coleman v1-0001-Doc

Re: Document spaces in .pgpass need to be escaped

2021-09-30 Thread James Coleman
On Wed, Sep 29, 2021 at 12:13 PM Tom Lane wrote: > > James Coleman writes: > > A coworker has a space in a Postgres password and noticed .pgpass > > didn't work; escaping it fixed the issue. That requirement wasn't > > documented (despite other escaping

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-15 Thread James Coleman
On Wed, Jul 14, 2021 at 9:22 PM David Rowley wrote: > > On Thu, 15 Jul 2021 at 12:30, Ranier Vilela wrote: > > > > Em qua., 14 de jul. de 2021 às 21:21, David Rowley > > escreveu: > >> But, in v8 there is no additional branch, so no branch to mispredict. > >> I don't really see how your explana

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-15 Thread James Coleman
On Thu, Jul 15, 2021 at 10:19 AM David Rowley wrote: > > On Fri, 16 Jul 2021 at 01:44, James Coleman wrote: > > > > On Wed, Jul 14, 2021 at 9:22 PM David Rowley wrote: > > > > > > On Thu, 15 Jul 2021 at 12:30, Ranier Vilela wrote: > > > > >

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-16 Thread James Coleman
On Thu, Jul 15, 2021 at 11:45 PM David Rowley wrote: > > On Fri, 16 Jul 2021 at 02:53, Ronan Dunklau wrote: > > Please find attached a v9 just moving the flag setting to ExecInitSort, and > > my > > apologies if I misunderstood your point. > > I took this and adjusted a few things and ended up w

Re: Early Sort/Group resjunk column elimination.

2021-07-16 Thread James Coleman
d the other examples you gave compelling also. Of course I haven't seen code yet, but my first intuition is to try to avoid adding extra nodes and teach the (hopefully few) relevant nodes to remove the resjunk entries themselves. Presumably in this case that would mostly be the sort nodes (including gather merge). One thing to pay attention to here is that we can't necessarily remove resjunk entries every time in a sort node since, for example, in parallel mode the gather merge node above it will need those entries to complete the sort. I'm interested to see what you're working on with a patch. Thanks, James Coleman

Re: Consider parallel for lateral subqueries with limit

2021-07-16 Thread James Coleman
On Thu, May 27, 2021 at 9:01 PM Greg Nancarrow wrote: > > On Tue, Dec 8, 2020 at 10:46 AM James Coleman wrote: > > > > While I haven't actually tracked down to guarantee this is handled > > elsewhere, a thought experiment -- I think -- shows it must be so. > >

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-19 Thread James Coleman
On Sat, Jul 17, 2021 at 4:36 AM David Rowley wrote: > > On Sat, 17 Jul 2021 at 01:14, James Coleman wrote: > > The only remaining question I have is whether or not costing needs to > > change, given the very significant speedup for datum sort. > > I'm looking a

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-20 Thread James Coleman
On Tue, Jul 20, 2021 at 4:35 AM David Rowley wrote: > > On Tue, 20 Jul 2021 at 01:10, James Coleman wrote: > > To be clear up front: I'm in favor of the patch, and I don't want to > > put unnecessary stumbling blocks up for it getting committed. So if we > > de

Re: Fix grammar oddities in comments

2024-06-10 Thread James Coleman
On Wed, Jun 5, 2024 at 5:34 AM David Rowley wrote: > > On Sun, 2 Jun 2024 at 10:08, James Coleman wrote: > > See attached for a small patch fixing some typos and grammatical > > errors in a couple of comments. > > Thanks. I pushed this after messing with the comm

Should we document how column DEFAULT expressions work?

2024-06-25 Thread James Coleman
possible I'm missing something. Thanks, James Coleman

Re: Should we document how column DEFAULT expressions work?

2024-06-25 Thread James Coleman
On Tue, Jun 25, 2024 at 4:59 PM Tom Lane wrote: > > James Coleman writes: > > It's possible I'm the only one who's been in this situation, but I've > > multiple times found myself explaining to a user how column DEFAULT > > expressions work: namel

Re: Should we document how column DEFAULT expressions work?

2024-07-01 Thread James Coleman
re in a kind of insidious way: the "bare" function call in DEFAULT is *not* executed as part of the query for DDL like it is with other queries. Hope this helps explain things. James Coleman

Re: RFC: Logging plan of the running query

2023-08-25 Thread James Coleman
On Thu, Aug 17, 2023 at 10:02 AM torikoshia wrote: > > On 2023-06-16 01:34, James Coleman wrote: > > Attached is v28 > > which sets ProcessLogQueryPlanInterruptActive to false in errfinish > > when necessary. Once built with those two patches I'm simply running > &

Re: RFC: Logging plan of the running query

2023-08-26 Thread James Coleman
On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: > > On Thu, Aug 17, 2023 at 10:02 AM torikoshia > wrote: > > > > On 2023-06-16 01:34, James Coleman wrote: > > > Attached is v28 > > > which sets ProcessLogQueryPlanInterruptActive to false in errfinis

Re: RFC: Logging plan of the running query

2023-08-28 Thread James Coleman
On Mon, Aug 28, 2023 at 3:01 AM torikoshia wrote: > > On 2023-08-26 21:03, James Coleman wrote: > > On Fri, Aug 25, 2023 at 7:43 AM James Coleman wrote: > >> > >> On Thu, Aug 17, 2023 at 10:02 AM torikoshia > >> wrote: > >> > > >> >

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-28 Thread James Coleman
t;, I'd far prefer to have that outcome then some automated process closing out my submission without my input when, as far as I can tell, the real problem is not my lack of activity by the required reviewers simply not looking at it. So I'm genuinely confused by you say "As long as this is not a surprise ending, I don't see the issue.". Perhaps we're imagining something different here reading between the lines? Regards, James Coleman

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-28 Thread James Coleman
erve to highlight patches that simply haven't had any review at all. I'd like to add a feature to the CF app that shows me my current patches by status, and I'd also like to have the option to have the CF app notify me when someone changes the status (I've noticed before that often a status gets changed without notification on list, and then I get surprised months later when it's stuck in "waiting on author"). Do either/both of those seem reasonable to add? Regards, James Coleman

Re: Add last_commit_lsn to pg_stat_database

2024-05-28 Thread James Coleman
nd 2. last_commit_lsn is not advancing, and 3. pg_current_wal_lsn() has advanced a lot then you can probably infer that there's a large amount of data that simply cannot be completed by the subscriber, and so there's no "real" delay. It also gives you an idea of how much data you will need to churn through (even if not replicated) once the transaction commits. Certainly understanding the data here will be simplest in the case where 1.) there's a single database and 2.) all tables are in the replication set, but I don't think the value is limited to that situation either. Regards, James Coleman

Re: Add last_commit_lsn to pg_stat_database

2024-05-28 Thread James Coleman
RecordTransactionCommitPrepared() for this to work, and, indeed, adding some logging to verify, the value of XactLastRecEnd we'd use to update XactLastCommitEnd is the same at the end of both of those functions during COMMIT PREPARED. I'd like to have someone weigh in on whether relying on

Fix grammar oddities in comments

2024-06-01 Thread James Coleman
of the comment opting to simply resolve the clear mistakes in the wording here. Regards, James Coleman v1-0001-Fix-comment-grammar-oddities.patch Description: Binary data

Fix broken link to FreeBSD DocProj in docs

2022-09-12 Thread James Coleman
See attached simple patch to fix $SUBJECT; the old link generates a Not Found. Thanks, James Coleman v1-0001-Fix-FreeBSD-DocProj-link.patch Description: Binary data

Re: Fix broken link to FreeBSD DocProj in docs

2022-09-13 Thread James Coleman
On Tue, Sep 13, 2022 at 4:43 AM Daniel Gustafsson wrote: > > > On 12 Sep 2022, at 20:46, Daniel Gustafsson wrote: > > > >> On 12 Sep 2022, at 18:13, James Coleman wrote: > > > >> See attached simple patch to fix $SUBJECT; the old link generates

Fix comment in convert_saop_to_hashed_saop

2022-09-14 Thread James Coleman
In 29f45e29 we added support for executing NOT IN(values) with a hashtable, however this comment still claims that we only do so for cases where the ScalarArrayOpExpr's useOr flag is true. See attached for fix. Thanks, James Coleman v1-0001-Fix-convert_saop_to_hashed_saop-comment.

Re: Consider parallel for lateral subqueries with limit

2022-09-19 Thread James Coleman
execution, then I believe your suggestion here is orthogonal and would expand the use cases even more. For example, if we were able to guarantee a unique result set (including order), then we could allow parallelizing subqueries even if they're not lateral and correlated. James Coleman

Support pg_attribute_aligned and noreturn in MSVC

2022-09-19 Thread James Coleman
I opted not to implement that attribute. James Coleman 1: https://www.postgresql.org/message-id/Yk6UgCGlZKuxRr4n%40paquier.xyz 2: 2008+ https://learn.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2008/k6ktzx3s(v=vs.90) 3. 2015+ https://learn.microsoft.com/en-us/cpp/c-language

Re: Support pg_attribute_aligned and noreturn in MSVC

2022-09-19 Thread James Coleman
On Mon, Sep 19, 2022 at 8:21 PM Michael Paquier wrote: > > On Mon, Sep 19, 2022 at 06:21:58PM -0400, James Coleman wrote: > > It turns out that MSVC supports both noreturn [2] [3] and alignment > > [4] [5] attributes, so this patch adds support for those. MSVC also > > su

Re: Support pg_attribute_aligned and noreturn in MSVC

2022-09-20 Thread James Coleman
On Mon, Sep 19, 2022 at 11:21 PM Michael Paquier wrote: > > On Mon, Sep 19, 2022 at 08:51:37PM -0400, James Coleman wrote: > > Yes, fixed. > > The CF bot is failing compilation on Windows: > http://commitfest.cputube.org/james-coleman.html > https://api.cirrus-ci.com/v1/ta

Auto explain after query timeout

2022-09-20 Thread James Coleman
impediment to doing so? Thanks, James Coleman

Re: Auto explain after query timeout

2022-09-20 Thread James Coleman
On Tue, Sep 20, 2022 at 2:12 PM Gurjeet wrote: > > On Tue Sep 20, 2022 at 10:34 AM PDT, James Coleman wrote: > > Hopefully I'm not missing something obvious, but as far as I know > > there's no way to configure auto explain to work fire > > statement_timeout f

Re: Auto explain after query timeout

2022-09-20 Thread James Coleman
On Tue, Sep 20, 2022 at 3:06 PM Robert Haas wrote: > > On Tue, Sep 20, 2022 at 2:35 PM James Coleman wrote: > > Either I'm missing something (and/or this was fixed in a later PG > > version), but I don't think this is how it works. We have this >

Re: Support pg_attribute_aligned and noreturn in MSVC

2022-09-21 Thread James Coleman
On Tue, Sep 20, 2022 at 9:18 PM Michael Paquier wrote: > > On Tue, Sep 20, 2022 at 08:01:20AM -0400, James Coleman wrote: > > I don't have access to a Windows machine for testing, but re-reading > > the documentation it looks like the issue is that our noreturn macr

Re: Consider parallel for lateral subqueries with limit

2022-09-22 Thread James Coleman
On Mon, Sep 19, 2022 at 4:29 PM Robert Haas wrote: > > On Mon, Sep 19, 2022 at 3:58 PM James Coleman wrote: > > But in the case where there's correlation via LATERAL we already don't > > guarantee unique executions for a given set of params into the lateral > &

Re: An inefficient query caused by unnecessary PlaceHolderVar

2023-05-30 Thread James Coleman
d rel is under the same lowest nulling outer + * join. I think this is clearer: "references something outside the subquery being pulled up and is not under the same lowest outer join." One other thing: it would be helpful to have the test query output be stable between HEAD and this patch; perhaps add: order by 1, 2, 3, 4, 5, 6, 7 to ensure stability? Thanks, James Coleman

Re: An inefficient query caused by unnecessary PlaceHolderVar

2023-06-01 Thread James Coleman
On Wed, May 31, 2023 at 10:30 PM Richard Guo wrote: > > > On Wed, May 31, 2023 at 1:27 AM James Coleman wrote: >> >> This looks good to me. > > > Thanks for the review! Sure thing! >> >> A few small tweaks suggested to comment wording: >> &g

Re: RFC: Logging plan of the running query

2023-06-02 Thread James Coleman
ning, and I think it would be worth adding that. Are you interested in re-opening this patch? I'd be happy to provide further review and help to try to push this along. I've rebased the patch and attached as v26. Thanks, James Coleman v26-0001-Add-function-to-log-the-plan-of-the-query.patch Description: Binary data

Re: RFC: Logging plan of the running query

2023-06-05 Thread James Coleman
On Mon, Jun 5, 2023 at 4:30 AM torikoshia wrote: > > On 2023-06-03 02:51, James Coleman wrote: > > Hello, > > > > Thanks for working on this patch! Sure thing! I'm *very interested* in seeing this available, and I think it paves the way for some additional features l

Add last_commit_lsn to pg_stat_database

2023-06-09 Thread James Coleman
) fits more naturally into the stats system. I'd originally thought of exposing it in pg_stat_wal, but that's per-cluster rather than per-database (indeed, this is a flaw I hadn't considered in the original patch), so I think pg_stat_database is the correct location. I've

Re: Parallelize correlated subqueries that execute within each worker

2023-06-11 Thread James Coleman
On Tue, Jun 6, 2023 at 4:36 AM Richard Guo wrote: > > > On Mon, Jan 23, 2023 at 10:00 PM James Coleman wrote: >> >> Which this patch we do in fact now see (as expected) rels with >> non-empty lateral_relids showing up in generate_[useful_]gather_paths. >> And t

Re: RFC: Logging plan of the running query

2023-06-12 Thread James Coleman
On Sun, Jun 11, 2023 at 11:07 PM torikoshia wrote: > > On 2023-06-06 03:26, James Coleman wrote: > > On Mon, Jun 5, 2023 at 4:30 AM torikoshia > > wrote: > >> > >> On 2023-06-03 02:51, James Coleman wrote: > >> > Hello, > >> > >

Re: RFC: Logging plan of the running query

2023-06-13 Thread James Coleman
On Tue, Jun 13, 2023 at 11:22 AM torikoshia wrote: > > On 2023-06-13 00:52, James Coleman wrote: > >> > >> > I've attached v27. The important change here in 0001 is that it > >> > guarantees the interrupt handler is re-entrant, since that was a bug >

Re: RFC: Logging plan of the running query

2023-06-14 Thread James Coleman
On Tue, Jun 13, 2023 at 11:53 AM James Coleman wrote: > > ... > I'm going to re-run tests with my patch version + resetting the flag > on SIGINT (and any other error condition) to be certain that the issue > you uncovered (where backends get stuck after a SIGINT not respondin

Re: RFC: Logging plan of the running query

2023-06-15 Thread James Coleman
On Thu, Jun 15, 2023 at 9:00 AM torikoshia wrote: > > On 2023-06-15 01:48, James Coleman wrote: > > On Tue, Jun 13, 2023 at 11:53 AM James Coleman > > wrote: > >> > >> ... > >> I'm going to re-run tests with my patch version + resetting the flag

path->param_info only set for lateral?

2023-06-18 Thread James Coleman
ateral reference) we're not going to get any ParamPathInfo added to the path or the rel. Is there a reason why we don't track the required relids providing the PARAM_EXEC params in this case? Thanks, James Coleman 1: https://www.postgresql.org/message-id/CAMbWs4_evjcMzN8Gw78bHfhfo2FKJThqhEjRJRmoMZx%3DNXcJ7w%40mail.gmail.com

Re: path->param_info only set for lateral?

2023-06-20 Thread James Coleman
On Sun, Jun 18, 2023 at 10:57 PM Tom Lane wrote: > > James Coleman writes: > > Over in "Parallelize correlated subqueries that execute within each > > worker" [1} Richard Guo found a bug in the current version of my patch > > in that thread. While debugging

Re: Use of additional index columns in rows filtering

2023-06-21 Thread James Coleman
Rows Removed by Index Recheck: 197780 > Filter: (b = 4) > Buffers: shared hit=544 > Planning Time: 0.105 ms > Execution Time: 13.690 ms > (10 rows) > > ... I did also confirm that this properly identifies cases Jeff had mentioned to me like "Index Filter: (((a * 2) > 50) AND ((b % 10) = 4))". I noticed also you still had questions/TODOs about handling index scans for join clauses. Regards, James Coleman 1: https://www.postgresql.org/message-id/20230609000600.syqy447e6metnvyj%40awork3.anarazel.de

Opportunistically pruning page before update

2023-06-21 Thread James Coleman
t had time to devote properly to that, so I'm wondering if there's anyone who might be interested in collaborating on that part. Other TODOs: - Audit other callers of RelationSetTargetBlock() to ensure they don't hold pointers into the page. Regards, James Coleman v1-000

Re: Use of additional index columns in rows filtering

2023-06-21 Thread James Coleman
On Wed, Jun 21, 2023 at 11:28 AM Tomas Vondra wrote: > > > > On 6/21/23 14:45, James Coleman wrote: > > Hello, > > > > I've cc'd Jeff Davis on this due to a conversation we had at PGCon > > about applying filters on index tuples during index scans.

Re: Memory leak in incremental sort re-scan

2023-06-21 Thread James Coleman
sorted_keys). That avoids unnecessary recreation of the sort states, but it also fixes the problem Tom noted as well: the call to preparePresortedCols() is already guarded by a test on fullsort_state being NULL, so with this change we also won't unnecessarily redo that work. Regards, James Coleman v2-0001-Fix-memory-leak-in-incremental-sort-rescan.patch Description: Binary data

Stampede of the JIT compilers

2023-06-23 Thread James Coleman
ered implemented a GUC/feature like "max_concurrent_jit_compilations" to cap the number of backends that may be compiling a query at any given point so that we avoid an optimization from running amok and consuming all of a servers resources? Regards, James Coleman

Re: Stampede of the JIT compilers

2023-06-24 Thread James Coleman
On Sat, Jun 24, 2023 at 7:40 AM Tomas Vondra wrote: > > > > On 6/24/23 02:33, David Rowley wrote: > > On Sat, 24 Jun 2023 at 02:28, James Coleman wrote: > >> There are a couple of issues here. I'm sure it's been discussed > >> before, and it'

Re: Stampede of the JIT compilers

2023-06-24 Thread James Coleman
On Sat, Jun 24, 2023 at 1:54 PM Tom Lane wrote: > > James Coleman writes: > > In that context capping the number of backends compiling, particularly > > where plans (and JIT?) might be cached, could well save us (depending > > on workload). > > TBH I do not find t

Re: Stampede of the JIT compilers

2023-06-24 Thread James Coleman
On Sat, Jun 24, 2023 at 8:14 PM David Rowley wrote: > > On Sun, 25 Jun 2023 at 05:54, Tom Lane wrote: > > > > James Coleman writes: > > > On Sat, Jun 24, 2023 at 7:40 AM Tomas Vondra > > > wrote: > > >> On 6/24/23 02:33, David Rowley wrote: >

Re: Stampede of the JIT compilers

2023-06-25 Thread James Coleman
does, but I don't > have a Google Cloud SQL or RDS instance running right to verify their > settings. I do seem to remember that they did as well though, at least a > while back. > > > Michael I believe it's off by default in Aurora Postgres also. Regards, James Coleman

Analyze on table creation?

2023-06-26 Thread James Coleman
so I assume people have considered it before. If so, I'd like to understand why the conclusion was not to do it, or, alternatively if it's a lack of tuits. Regards, James Coleman

Re: Analyze on table creation?

2023-06-26 Thread James Coleman
On Mon, Jun 26, 2023 at 1:45 PM Pavel Stehule wrote: > > > > po 26. 6. 2023 v 19:43 odesílatel Pavel Stehule > napsal: >> >> Hi >> >> po 26. 6. 2023 v 19:41 odesílatel James Coleman napsal: >>> >>> Hello, >>> >>> Have w

Re: Analyze on table creation?

2023-06-26 Thread James Coleman
cc'ing Tom because I'm curious if he's willing to provide some greater context on the commit in question. On Mon, Jun 26, 2023 at 2:16 PM Pavel Stehule wrote: > > > > po 26. 6. 2023 v 19:48 odesílatel James Coleman napsal: >> >> On Mon, Jun 26, 2

Re: Analyze on table creation?

2023-06-26 Thread James Coleman
On Mon, Jun 26, 2023 at 4:00 PM Andres Freund wrote: > > Hi, > > On 2023-06-26 13:40:49 -0400, James Coleman wrote: > > Have we ever discussed running an analyze immediately after creating a > > table? > > That doesn't make a whole lot of sense to me - we coul

Re: Analyze on table creation?

2023-06-27 Thread James Coleman
On Mon, Jun 26, 2023 at 4:16 PM James Coleman wrote: > > On Mon, Jun 26, 2023 at 4:00 PM Andres Freund wrote: > > > > Hi, > > > > On 2023-06-26 13:40:49 -0400, James Coleman wrote: > > > Have we ever discussed running an analyze immediately after creati

pgindent (probably my missing something obvious)

2023-07-03 Thread James Coleman
aving some very long function calls. I've downloaded the latest typedefs list, but I haven't added any types anyway. What obvious thing am I missing? Thanks, James Coleman

Re: Parallelize correlated subqueries that execute within each worker

2023-07-03 Thread James Coleman
On Sun, Jun 11, 2023 at 10:23 PM James Coleman wrote: > > ... > > And while trying the v9 patch I came across a crash with the query > > below. > > > > set min_parallel_table_scan_size to 0; > > set parallel_setup_cost to 0; > > set parallel_tuple_cost to

Re: pgindent (probably my missing something obvious)

2023-07-03 Thread James Coleman
On Mon, Jul 3, 2023 at 9:20 PM Tom Lane wrote: > > James Coleman writes: > > This is the first time I've run pgindent on my current machine, and it > > doesn't seem to be making any modifications to source files. For > > example this command: > > >

Seq scan instead of index scan querying single row from primary key on large table

2024-07-18 Thread James Coleman
7;s true I think the early return cost multiplication of the LIMIT is being applied very naively on the seq scan node. Or perhaps the issue is that the startup cost for a single tuple on a seq scan like this shouldn't really have a startup cost of 0 -- that cost is presumably for tuples being returned _without_ having applied the filter. That seems slightly odd to me, because the cost of getting the first row out of that node -- in my naive view thinking about it for all of 5 seconds -- should be calculated based on applying the filter (and thus the likelihood that that filter matches right away). If we did that then this path would never win. But that 0.00 startup cost for the seq scan with a filter shows up in PG14 and PG11 also, not just PG16, so that's not something that's changed. To recap: the estimation of rows is correct, the estimated high (total) cost of the seq scan is correct, but the seq scan is chosen over the index scan anyway for a plan that returns a single "random" row based on the primary key. Am I right to be surprised here? James Coleman

Re: Seq scan instead of index scan querying single row from primary key on large table

2024-07-18 Thread James Coleman
On Thu, Jul 18, 2024 at 2:38 PM Tom Lane wrote: > > James Coleman writes: > > The plan generated by the planner changed suddenly one morning this > > week, and in a very surprising way: the innermost scan (of "objects") > > started choosing a seq scan, des

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-06-24 Thread James Coleman
On Fri, Jun 19, 2020 at 12:04 AM Justin Pryzby wrote: > > On Tue, Apr 07, 2020 at 08:40:30AM -0400, James Coleman wrote: > > On Tue, Apr 7, 2020 at 12:25 AM Justin Pryzby wrote: > > > On Mon, Apr 06, 2020 at 09:57:22PM +0200, Tomas Vondra wrote: > > > > I'

Re: Open Item: Should non-text EXPLAIN always show properties?

2020-06-25 Thread James Coleman
On Thu, Jun 25, 2020 at 5:15 AM David Rowley wrote: > > Over on [1] Justin mentions that the non-text EXPLAIN ANALYZE should > always show the "Disk Usage" and "HashAgg Batches" properties. I > agree with this. show_wal_usage() is a good example of how we normally > do things. We try to keep the

Re: Open Item: Should non-text EXPLAIN always show properties?

2020-06-25 Thread James Coleman
On Thu, Jun 25, 2020 at 12:33 PM Tom Lane wrote: > > Robert Haas writes: > > On Thu, Jun 25, 2020 at 8:42 AM James Coleman wrote: > >> Yesterday I'd replied [1] to Justin's proposal for this WRT > >> incremental sort and expressed my opinion tha

Re: suggest to rename enable_incrementalsort

2020-07-02 Thread James Coleman
I think the change makes a lot of sense. The only reason I had it as enable_incrementalsort in the first place was trying to broadly following the existing GUC names, but as has already been pointed out, there's a lot of variation there, and my version of the patch already changed it to be more rea

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-07-02 Thread James Coleman
It seems like the consensus over at another discussion on this topic [1] is that we ought to go ahead and print the zeros [for machine readable output formats], even though that creates some interesting scenarios like the fact that disk sorts will print 0 for memory even though that's not true. Th

Re: Use of "long" in incremental sort code

2020-07-02 Thread James Coleman
On Tue, Jun 30, 2020 at 7:21 AM Peter Eisentraut wrote: > > On 2020-06-30 06:24, David Rowley wrote: > > On Tue, 30 Jun 2020 at 16:20, Tom Lane wrote: > >> There is a fairly widespread issue that memory-size-related GUCs and > >> suchlike variables are limited to represent sizes that fit in a "lo

Re: Use of "long" in incremental sort code

2020-07-02 Thread James Coleman
On Thu, Jul 2, 2020 at 1:36 PM Peter Geoghegan wrote: > > On Mon, Jun 29, 2020 at 9:13 PM David Rowley wrote: > > I noticed the incremental sort code makes use of the long datatype a > > few times, e.g in TuplesortInstrumentation and > > IncrementalSortGroupInfo. > > I agree that long is terrible

Re: Use of "long" in incremental sort code

2020-07-02 Thread James Coleman
On Thu, Jul 2, 2020 at 3:39 PM Tom Lane wrote: > > Peter Geoghegan writes: > > On Thu, Jul 2, 2020 at 10:53 AM James Coleman wrote: > >> Do you think it's reasonable to use int64 across the board for memory > >> and disk space numbers then? If so, I can updat

Re: Patch for nodeIncrementalSort comment correction.

2020-07-19 Thread James Coleman
On Saturday, July 18, 2020, vignesh C wrote: > Hi, > > One of the comments needs correction "sorting all tuples in the the > dataset" should have been "sorting all tuples in the dataset". > The Attached patch has the changes for the same. > > Regards, > Vignesh > EnterpriseDB: http://www.enterpri

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 7:31 PM Tomas Vondra wrote: > > On Mon, Apr 06, 2020 at 07:09:11PM -0400, James Coleman wrote: > >On Mon, Apr 6, 2020 at 6:13 PM Tomas Vondra > > wrote: > >> > >> On Mon, Apr 06, 2020 at 05:47:48PM -0400, James Coleman wrote: > &

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 9:46 PM Tom Lane wrote: > > Tomas Vondra writes: > > I don't know, I've tried running the tests on a number of machines, > > similar to those failing. Rapsberry Pi, Fedora 31, ... and it worked > > everywhere while the failures seem consistent. > > On my machine, it reprodu

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2020-04-06 Thread James Coleman
On Mon, Apr 6, 2020 at 10:09 PM Tom Lane wrote: > > James Coleman writes: > > On Mon, Apr 6, 2020 at 9:46 PM Tom Lane wrote: > >> I think the correct fix is to change the enum declaration. > > > Hmm. I don't actually really like that, because it mean

<    1   2   3   4   5   6   >