On Mon, Mar 9, 2020 at 3:56 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> Assuming we'll implement it in a way that we do not know about what kind
> of path type is that in create_distinct_path, then it can also work for
> ProjectionPath or anything else (if UniqueKeys are present). But then
On Mon, Mar 9, 2020 at 6:28 PM Andres Freund wrote:
>
> Hi,
>
> On 2020-03-08 20:12:21 -0400, James Coleman wrote:
> > I recently noticed while setting up a test environment that attempting to
> > connect to a standby running without hot_standby=on results in a fairly
>
On Mon, Mar 9, 2020 at 8:06 PM Andres Freund wrote:
>
> Hi,
>
> On 2020-03-09 18:40:32 -0400, James Coleman wrote:
> > On Mon, Mar 9, 2020 at 6:28 PM Andres Freund wrote:
> > > > I wanted to get some initial feedback on the idea before writing a
> > >
s here, and I realized that my test case
from far upthread is actually a useful setup to see how much overhead
is involved in sorting each batch individually, since it sets up data
with each batch only containing 1 tuple. That particular case is one
we could easily optimize anyway in the code and skip sorting
altogether -- might be a useful enhancement.
I hope to do some more testing and then report back with the results.
James Coleman
also, but likely won't get to it until later this week or next week at
the earliest.
Jesper: Is there anything still on your list of things to change about
the patch? Or would now be a good time to look hard at the code?
James Coleman
e: this also means the function header comment "Must be called
before inserting any tuples" is a condition that isn't actually
validated, but I think that's fine given it's not a new problem and
even more so since the same comment goes on to say that
ith more
detailed comments on my current benchmarking work.
James Coleman
incremental-sort-28.patch
Description: Binary data
On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
>I think the first thing to do is get some concrete numbers on performance if
>we:
>
>1. Only sort one group at a time.
>2. Update the costing to prefer traditional sort unless we have very
>high confidence we'
On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs wrote:
>
> On Mon, 24 Jun 2019 at 16:10, James Coleman wrote:
>>
>> On Thu, Jun 13, 2019 at 11:38:12PM -0400, James Coleman wrote:
>> >I think the first thing to do is get some concrete numbers on performance
>>
On Mon, Jun 24, 2019 at 4:16 PM Tomas Vondra
wrote:
>
> On Mon, Jun 24, 2019 at 01:00:50PM -0400, James Coleman wrote:
> >On Mon, Jun 24, 2019 at 12:56 PM Simon Riggs wrote:
> >>
> >> On Mon, 24 Jun 2019 at 16:10, James Coleman wrote:
> >>>
> >&g
On Tue, Jun 25, 2019 at 12:02 PM Tomas Vondra
wrote:
>
> On Mon, Jun 24, 2019 at 07:34:19PM -0400, James Coleman wrote:
> >On Mon, Jun 24, 2019 at 4:16 PM Tomas Vondra
> > wrote:
> >>
> >> On Mon, Jun 24, 2019 at 01:00:50PM -0400, James Coleman wrote:
> >
On Tue, Jun 25, 2019 at 1:13 PM Peter Geoghegan wrote:
>
> On Tue, Jun 25, 2019 at 9:53 AM James Coleman wrote:
> > Given the patch contents I don't see any obviously reason why either
> > of those possibilities (calling comparetup_heap less often, or it's
>
On Tue, Jun 25, 2019 at 4:32 PM Tomas Vondra
wrote:
>
> On Tue, Jun 25, 2019 at 12:13:01PM -0700, Peter Geoghegan wrote:
> >On Tue, Jun 25, 2019 at 11:03 AM James Coleman wrote:
> >> No, I haven't confirmed that it's called less frequently, and I'd be
>
g something obvious, and in that case, please
> ignore.
I think that example is the opposite direction of what David (Rowley)
is saying. Unique on {a, b} implies unique on {a, b, c} while you're
correct that the inverse doesn't hold.
Unique on {a, b} also implies unique on {b, a} as well as on {b, a, c}
and {c, a, b} and {c, b, a} and {a, c, b}, which is what makes this
different from pathkeys.
James Coleman
On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
wrote:
>
> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
> >
> >Unrelated: if you or someone else you know that's more familiar with
> >the parallel code, I'd be interested in their looking at the patc
On Thu, Jul 4, 2019 at 10:46 AM Tomas Vondra
wrote:
>
> On Thu, Jul 04, 2019 at 09:29:49AM -0400, James Coleman wrote:
> >On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
> > wrote:
> >>
> >> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
> >&
On Sun, Jul 7, 2019 at 8:34 AM Tomas Vondra
wrote:
>
> On Thu, Jul 04, 2019 at 09:29:49AM -0400, James Coleman wrote:
> >On Tue, Jun 25, 2019 at 7:22 PM Tomas Vondra
> > wrote:
> >>
> >> On Tue, Jun 25, 2019 at 04:53:40PM -0400, James Coleman wrote:
> >&
we're discussing, but it seems to
me that it at least fits more with what the comments imply.
I'll try to look at it a bit more later also, but at the moment other
work calls.
James Coleman
On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
wrote:
>
> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
> >On Sun, Jul 7, 2019 at 5:02 PM Tomas Vondra
> > wrote:
> >> We're running query like this:
> >>
> >> SELECT a, sum(b), coun
On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra
wrote:
>
> On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
> >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondra
> > wrote:
> >>
> >> On Mon, Jul 08, 2019 at 09:22:39AM -0400, James Coleman wrote:
> &
apture the debugging process for reference.
On Mon, Jul 8, 2019 at 12:07 PM James Coleman wrote:
>
> On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra
> wrote:
> >
> > On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
> > >On Mon, Jul 8, 2019 at 9:59 AM Tomas Vondr
On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
wrote:
>
> On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote:
> >On Mon, Jul 8, 2019 at 10:58 AM Tomas Vondra
> > wrote:
> >>
> >> On Mon, Jul 08, 2019 at 10:32:18AM -0400, James Coleman wrote:
> &
On Mon, Jul 8, 2019 at 6:37 PM Alexander Korotkov
wrote:
>
> On Thu, Jul 4, 2019 at 4:25 PM James Coleman wrote:
> > Process questions:
> > - Do I need to explicitly move the patch somehow to the next CF?
>
> We didn't manage to register it on current (July) commit
On Tue, Jul 9, 2019 at 10:54 AM Tomas Vondra
wrote:
>
> On Tue, Jul 09, 2019 at 09:28:42AM -0400, James Coleman wrote:
> >On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
> > wrote:
> >>
> >> On Mon, Jul 08, 2019 at 12:07:06PM -0400, James Coleman wrote:
> >
>
> >Tangentially: I'd almost expect enable_incremental_sort to act as a
> >hard disable (and not even generate the paths) rather than a soft
> >cost-based disable, since while standard sort is the most basic
> >operation that needs to always be available as a last resort the same
> >is not true for incremental sort...
> >
>
> Good point. It's somewhat similar to options like enable_parallel_hash
> which also are "hard" switches (i.e. not cost-based penalties).
I assume the proper approach here then is to check the GUC before
building and adding the path?
> >If we end up wanting to limit the number of places we consider
> >incremental sort (whether for planning time or merely for size of the
> >initial patch, do you have any thoughts on what general areas we
> >should consider most valuable? Besides the obvious LIMIT case aother
> >area that might benefit was min/max, though I'm not sure yet at the
> >moment if that would really end up meaning considering it all over the
> >place.
> >
>
> OK, sounds like a plan!
Did you have any thoughts on the question about where this is likely
to be most valuable?
James Coleman
On Mon, Jul 8, 2019 at 9:37 PM Tomas Vondra
wrote:
> Now, consider this example:
>
> create table t (a int, b int, c int);
> insert into t select mod(i,100),mod(i,100),i from
> generate_series(1,1000) s(i);
> create index on t (a);
> analyze t;
> explain select a,b,sum(c) from t gro
Workers Planned: 4
-> Partial GroupAggregate
Group Key: a, b
-> Sort
Sort Key: a, b
-> Parallel Seq Scan on t
(or if I disable seqscan then the sort+seq scan above becomes inc sort
+ index scan)
To be honest, I don't think I understand how you would get a plan like
that anyway since the index here only has "a" and so can't provide
(pathkeys: a, b).
Perhaps there's something I'm still missing though.
James Coleman
On Sat, Jul 20, 2019 at 12:12 PM James Coleman wrote:
>
> On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
> wrote:
> ...
> > >My current line of investigation is whether we need to do anything in
> > >the parallel portion of create_ordered_paths(). I noticed that th
On Sat, Jul 20, 2019 at 1:00 PM Tomas Vondra
wrote:
>
> On Sat, Jul 20, 2019 at 12:21:01PM -0400, James Coleman wrote:
> >On Sat, Jul 20, 2019 at 12:12 PM James Coleman wrote:
> >>
> >> On Sat, Jul 20, 2019 at 11:25 AM Tomas Vondra
> >> wrote:
> >&
On Tue, Sep 8, 2020 at 4:37 PM Heikki Linnakangas wrote:
>
> On 08/09/2020 22:25, James Coleman wrote:
> > On Wed, Aug 19, 2020 at 3:16 AM Heikki Linnakangas wrote:
> >>
> >> You could also apply the optimization for non-Const expressions, as lo
If there's a better list than this, please let me know, but I figured
hackers is appropriate since the extension building infrastructure is
documented in core.
While working on an in-house extension I realized that while PGXS
provides the standard regression test infrastructure, I'm not aware of
a
On Mon, Sep 21, 2020 at 11:36 AM Tom Lane wrote:
>
> James Coleman writes:
> > If there's a better list than this, please let me know, but I figured
> > hackers is appropriate since the extension building infrastructure is
> > documented in core.
>
> > Wh
On Thu, Sep 24, 2020 at 9:30 AM Jonathan S. Katz wrote:
>
> I am pleased to report that PostgreSQL 13 is now GA:
>
> https://www.postgresql.org/about/news/2077/
I'm not sure if there was a "release announcement draft" thread
(searching my folder didn't find one for GA, but it's also easily
possib
I'm breaking out a few questions I'd posed on another thread about the
release timeline [1] into this new thread.
I noticed on the PG13 release announcement that the link for
incremental sort goes to the GUC docs [2] because (as Jonathan Katz
confirmed in the linked thread) that page actually has
On Sat, Sep 26, 2020 at 2:49 PM Jaime Casanova
wrote:
>
> Hi,
>
> With sqlsmith I found a query that gives this error:
> ERROR: ORDER/GROUP BY expression not found in targetlist
>
> I noted the query (sql query below, sorry it uses custom tables i
> couldn't replicate with regression tables) beca
On Thu, Oct 1, 2020 at 3:09 AM Jaime Casanova
wrote:
>
> On Wed, 30 Sep 2020 at 21:21, James Coleman wrote:
> >
> > On Sat, Sep 26, 2020 at 2:49 PM Jaime Casanova
> > wrote:
> > >
> > > Hi,
> > >
> > > With sqlsmith I found a query th
On Thu, Oct 1, 2020 at 6:08 PM Tomas Vondra
wrote:
>
> On Thu, Oct 01, 2020 at 09:02:57AM -0400, James Coleman wrote:
> >On Thu, Oct 1, 2020 at 3:09 AM Jaime Casanova
> > wrote:
> >>
> >> On Wed, 30 Sep 2020 at 21:21, James Coleman wrote:
> >> &
On Thu, Oct 1, 2020 at 9:10 PM James Coleman wrote:
>
> On Thu, Oct 1, 2020 at 6:08 PM Tomas Vondra
> wrote:
> >
> > On Thu, Oct 01, 2020 at 09:02:57AM -0400, James Coleman wrote:
> > >On Thu, Oct 1, 2020 at 3:09 AM Jaime Casanova
> > > wrote:
> >
On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra
wrote:
>
> On Fri, Oct 02, 2020 at 09:19:44AM -0400, James Coleman wrote:
> >
> > ...
> >
> >I've been able to confirm that the problem goes away if we stop adding
> >the gather merge paths in generate_usefu
On Fri, Oct 2, 2020 at 10:53 AM James Coleman wrote:
>
> On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra
> wrote:
> >
> > On Fri, Oct 02, 2020 at 09:19:44AM -0400, James Coleman wrote:
> > >
> > > ...
> > >
> > >I've been able to confir
On Fri, Oct 2, 2020 at 2:25 PM Tomas Vondra
wrote:
>
> On Fri, Oct 02, 2020 at 10:55:14AM -0400, James Coleman wrote:
> >On Fri, Oct 2, 2020 at 10:53 AM James Coleman wrote:
> >>
> >> On Fri, Oct 2, 2020 at 10:32 AM Tomas Vondra
> >> wrote:
> >> &
On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra
wrote:
>
> On Fri, Oct 02, 2020 at 04:12:11PM -0400, James Coleman wrote:
> >On Fri, Oct 2, 2020 at 2:25 PM Tomas Vondra
> > wrote:
> >>
> >> On Fri, Oct 02, 2020 at 10:55:14AM -0400, James Coleman wrote:
> &g
On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra
wrote:
> >And I don't see any reason why the CASE statement couldn't in theory
> >(I don't know the internals enough to know when it actually happens)
> >be done as part of the base relation scan (in this case, the seq
> >scan). It's not dependent on any
On Fri, Oct 2, 2020 at 6:28 PM Tomas Vondra
wrote:
>
> On Fri, Oct 02, 2020 at 05:45:52PM -0400, James Coleman wrote:
> >On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra
> > wrote:
> >>
> >> ...
> >>
> >> More importanly, it does not actually fix t
On Fri, Oct 2, 2020 at 7:07 PM James Coleman wrote:
>
> On Fri, Oct 2, 2020 at 6:28 PM Tomas Vondra
> wrote:
> >
> > On Fri, Oct 02, 2020 at 05:45:52PM -0400, James Coleman wrote:
> > >On Fri, Oct 2, 2020 at 4:56 PM Tomas Vondra
> > > wrote:
> > &g
On Fri, Oct 2, 2020 at 2:25 PM Tomas Vondra
wrote:
>
> The backtrace looks like this:
>
> #0 get_sortgroupref_tle
> #1 0x00808ab9 in prepare_sort_from_pathkeys
> #2 0x0080926c in make_sort_from_pathkeys
> #3 0x00801032 in create_sort_plan
> #4
On Fri, Oct 2, 2020 at 11:16 PM James Coleman wrote:
>
> On Fri, Oct 2, 2020 at 7:07 PM James Coleman wrote:
> >
> > On Fri, Oct 2, 2020 at 6:28 PM Tomas Vondra
> > wrote:
> > >
> > > On Fri, Oct 02, 2020 at 05:45:52PM -0400, James Coleman wrote:
&
On Sat, Oct 3, 2020 at 5:44 PM Tomas Vondra
wrote:
>
> On Sat, Oct 03, 2020 at 10:50:06AM -0400, James Coleman wrote:
> >On Fri, Oct 2, 2020 at 11:16 PM James Coleman wrote:
> >>
> >> On Fri, Oct 2, 2020 at 7:07 PM James Coleman wrote:
> >> >
> >
On Sat, Oct 3, 2020 at 10:10 PM James Coleman wrote:
>
> On Sat, Oct 3, 2020 at 5:44 PM Tomas Vondra
> wrote:
> >
> > On Sat, Oct 03, 2020 at 10:50:06AM -0400, James Coleman wrote:
> > >On Fri, Oct 2, 2020 at 11:16 PM James Coleman wrote:
> > >>
&
On Sun, Oct 4, 2020 at 9:40 PM James Coleman wrote:
>
> On Sat, Oct 3, 2020 at 10:10 PM James Coleman wrote:
> >
> > On Sat, Oct 3, 2020 at 5:44 PM Tomas Vondra
> > wrote:
> > >
> > > On Sat, Oct 03, 2020 at 10:50:06AM -0400, James Coleman wrote:
&g
On Mon, Oct 5, 2020 at 11:33 AM James Coleman wrote:
>
> On Sun, Oct 4, 2020 at 9:40 PM James Coleman wrote:
> >
> > On Sat, Oct 3, 2020 at 10:10 PM James Coleman wrote:
> > >
> > > On Sat, Oct 3, 2020 at 5:44 PM Tomas Vondra
> > > wrote:
>
On Mon, Oct 5, 2020 at 10:38 PM James Coleman wrote:
>
> On Mon, Oct 5, 2020 at 11:33 AM James Coleman wrote:
> >
> > On Sun, Oct 4, 2020 at 9:40 PM James Coleman wrote:
> > >
> > > On Sat, Oct 3, 2020 at 10:10 PM James Coleman wrote:
> > > >
>
On Tue, Oct 6, 2020 at 9:28 AM Jaime Casanova
wrote:
> Can you please create an entry in the commitfest for this one so we
> don't lose track of it?
Done.
On Wed, Oct 7, 2020 at 3:52 PM Robert Haas wrote:
>
> On Thu, Oct 1, 2020 at 9:03 AM James Coleman wrote:
> > The plan (obtained by replacing the volatile function with a stable one):
> >
> > Unique
> >-> Nested Loop
> > -> Gather M
On Wed, Oct 7, 2020 at 6:22 PM Tomas Vondra
wrote:
>
> On Wed, Oct 07, 2020 at 04:01:27PM -0400, James Coleman wrote:
> >On Wed, Oct 7, 2020 at 3:52 PM Robert Haas wrote:
> >>
> >> On Thu, Oct 1, 2020 at 9:03 AM James Coleman wrote:
> >> > The plan (o
I've been wondering recently why the external canonical form of types
like char and varchar doesn't match the typname in pg_type.
Additionally, the alternative/extended names are hardcoded in
format_type.c rather than being an additional column in that catalog
table.
I would have assumed there wer
I've attached an updated copy of the patch that applies cleanly to
current master.
commit 6428245702a40b3e3fa11bb64b7611cdd33a0778
Author: Alexander Korotkov
Date: Sat Apr 7 18:51:20 2018 +0300
Implement incremental sort
Incremental sort is an optimized variant of multikey sort
The incremental sort patch seems to significantly improve performance for
your query: https://commitfest.postgresql.org/17/1124/
On Fri, Jun 1, 2018 at 7:46 AM, Christopher Wilson <
chris.wil...@cantabcapital.com> wrote:
> Dear Postgres developers,
>
>
>
> I sent this query to the performance lis
The following review has been posted through the commitfest application:
make installcheck-world: tested, failed
Implements feature: tested, passed
Spec compliant: not tested
Documentation:not tested
A fairly common planning problem for us is what we call "most recent
6, 2018 at 10:39 AM Tom Lane wrote:
> James Coleman writes:
> > A fairly common planning problem for us is what we call "most recent
> first" queries; i.e., "the 50 most recent rows for a ".
>
> > Here's a basic setup:
>
> > -- created_at has
Added to TODO:
Add function to allow easier transaction id comparisons
http://archives.postgresql.org/pgsql-hackers/2011-11/msg00786.php
Did this ever happen? Or did it fall through the cracks?
since the
change to default to trying to inline CTEs rather than defaulting to
materializing them.
Attached is a patch to slightly modify the language; would be happy to
hear suggestions on a better way to improve this.
Regards,
James Coleman
v1-0001-Update-parallel-safety-docs-for-CTE-optimizat
iO3dfUnVmBs4M-4aUX_zHmPN72ELE7c_8qAO_toPmA%40mail.gmail.com
>
> Kind Regards,
> Peter Smith.
> Fujitsu Australia
Yes, I would appreciate it moving forward.
Did you want to create the CF entry or should I?
Regards,
James Coleman
vel is something that would be a reasonable solution
to this problem, or if folks think there's a better way to solve the
problem.
Regards,
James Coleman
On Tue, Nov 19, 2024 at 2:16 PM James Coleman wrote:
>
> Hello,
>
> A colleague noticed today that the docs still say that "Scans of
> common table expressions (CTEs)" are "always parallel restricted".
>
> While I think that strictly remains true at
501 - 565 of 565 matches
Mail list logo
