Re: Postgres with pthread

2017-12-10 Thread james
On 06/12/2017 17:26, Andreas Karlsson wrote: An additional issue is that this could break a lot of extensions and in a way that it is not apparent at compile time. This means we may need to break all extensions to force extensions authors to check if they are thread safe. I do not like making

Re: Postgres with pthread

2017-12-27 Thread james
> All threads are blocked in semaphores. That they are blocked is inevitable - I guess the issue is that they are thrashing. I guess it would be necessary to separate the internals to have some internal queueing and effectively reduce the number of actively executing threads. In effect make th

Re: Postgres with pthread

2017-12-27 Thread james
On 27/12/2017 10:08, Andres Freund wrote: Optimizing for this seems like a pointless exercise. If the goal is efficient processing of 100k connections the solution is a session / connection abstraction and a scheduler. Optimizing for this amount of concurrency just will add complexity and sl

Re: Synchronizing slots from primary to standby

2022-02-23 Thread James Coleman
ccur without any attempt to trigger the catalog xmin problem. Given this error seems pretty significant in terms of indicating fundamental lack of test coverage (the primary stated benefit of the patch is physical failover), and it currently is a blocker to testing more deeply. Thanks, Jame

Consider parallel for lateral subqueries with limit

2020-11-30 Thread James Coleman
about the safety of this (or any other issue), I'd appreciate the feedback. James From 0aff5f1b5e35e37a311c01e9f53caf6e088e8d43 Mon Sep 17 00:00:00 2001 From: jcoleman Date: Mon, 30 Nov 2020 11:36:35 -0500 Subject: [PATCH v1] Allow parallel LATERAL subqueries with LIMIT/OFFSET The code that

Re: enable_incremental_sort changes query behavior

2020-11-30 Thread James Coleman
t;undo" (or if that's desirable) the move to the next CF. Thoughts? James 1: https://commitfest.postgresql.org/30/2754/

Re: [DOC] Document concurrent index builds waiting on each other

2020-11-30 Thread James Coleman
ckground of the system, not just something people trigger manually, and 2.) we ought to document things where the user action triggering the behavior is documented. > I'm not sure on the wording to use; what about this? The wording seems fine to me. This is a replacement for what was 0002 earlier? And 0001 from earlier still seems to be a useful standalone patch? James

Re: Why does create_gather_merge_plan need make_sort?

2020-11-30 Thread James Coleman
On Sun, Nov 29, 2020 at 4:10 PM Tomas Vondra wrote: > > > > On 11/29/20 3:44 PM, James Coleman wrote: > > On Mon, Nov 23, 2020 at 8:19 AM James Coleman wrote: > >> > >> .. > >> > >> So I'm +1 on turning it into an ERROR log instead,

Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-11-30 Thread James Coleman
On Sun, Nov 29, 2020 at 4:20 PM Tomas Vondra wrote: > > On 11/29/20 3:26 PM, James Coleman wrote: > > On Mon, Nov 23, 2020 at 8:35 AM James Coleman wrote: > >> > >> On Sun, Nov 22, 2020 at 4:59 PM Tomas Vondra > >> wrote: > >>> ... > >>

Re: Consider parallel for lateral subqueries with limit

2020-12-01 Thread James Coleman
On Mon, Nov 30, 2020 at 7:00 PM James Coleman wrote: > > I've been investigating parallelizing certain correlated subqueries, > and during that work stumbled across the fact that > set_rel_consider_parallel disallows parallel query on what seems like > a fairly simple cas

Re: [DOC] Document concurrent index builds waiting on each other

2020-12-01 Thread James Coleman
On Tue, Dec 1, 2020 at 6:51 PM Alvaro Herrera wrote: > > On 2020-Nov-30, James Coleman wrote: > > > On Mon, Nov 30, 2020 at 4:53 PM Alvaro Herrera > > wrote: > > > > > > On 2020-Sep-30, Michael Paquier wrote: > > > > Yeah, I

Re: Change definitions of bitmap flags to bit-shifting style

2020-12-06 Thread James Coleman
ce-conventions.html? > -- > Michael In my view the bit shifting approach makes it more obvious a single bit is being set, but on the other hand the hex approach makes it easier to compare in debugging. I’m not really sure which to prefer, though I think I would have leaned slightly towards the former. James >

Re: Consider parallel for lateral subqueries with limit

2020-12-07 Thread James Coleman
27;ll be the first to admit that it's easy enough for me to miss a key > piece of logic on something like this, so I could be way off base too. If it weren't for the volatile function in the example, I think I could argue we could change before (i.e., my theorizing above originally ab

Re: Insert Documentation - Returning Clause and Order

2020-12-12 Thread James Coleman
It’s not so much about ordering, per se, but about identity. Certainly almost every ORM, and maybe even other forms of application code, need to be able to associate the serial column value returned with what it inserted. I'd expect something like that (whether by ordering explicitly or by providing some kind of mapping between indexes in the statement data and the inserted/returned row values). James

Re: Insert Documentation - Returning Clause and Order

2020-12-12 Thread James Coleman
On Sat, Dec 12, 2020 at 10:11 AM David G. Johnston wrote: > > On Sat, Dec 12, 2020 at 7:02 AM James Coleman wrote: >> >> >> Certainly almost every ORM, and maybe even other forms of application >> code, need to be able to associate the serial column value ret

Re: [DOC] Document concurrent index builds waiting on each other

2020-12-18 Thread James Coleman
On Tue, Dec 1, 2020 at 8:05 PM James Coleman wrote: > > On Tue, Dec 1, 2020 at 6:51 PM Alvaro Herrera wrote: > > > > On 2020-Nov-30, James Coleman wrote: > > > > > On Mon, Nov 30, 2020 at 4:53 PM Alvaro Herrera > > > wrote: > > &

Re: Dependency isn't created between extension and schema

2020-12-21 Thread James Coleman
exist. > While on it.. The docs are incorrect here. As you say, > CreateExtensionInternal() may internally create a schema. Alternatively the behavior could be updated to match the docs, since that seems like reasonable intent. James

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-05 Thread James Coleman
On Fri, Mar 5, 2021 at 12:36 PM Fujii Masao wrote: > > > > On 2021/03/05 22:45, David Steele wrote: > > Hi Fujii, > > > > On 9/8/20 1:17 PM, James Coleman wrote: > >> On Tue, Aug 18, 2020 at 12:25 PM Fujii Masao > >> wrote: > >>> Thank

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-09 Thread James Coleman
t; > > that. This looks a bit overkill to me against the purpose, at least for > > > me. > > > > Agreed. If hot standby is off, why would the admin care about whether > > it's consistent yet or not? > > Great, so we're agreed on the messages to emit. James, are you updating > your patch, considering Fujii's note about the new signal and pmstate > that need to be added? Perhaps I'm missing something, but I was under the impression the "prefer the former message" meant we were not adding a new signal and pmstate? James

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-09 Thread James Coleman
On Tue, Mar 9, 2021 at 9:07 AM Alvaro Herrera wrote: > > On 2021-Mar-09, James Coleman wrote: > > > On Tue, Mar 9, 2021 at 8:47 AM Alvaro Herrera > > wrote: > > > > > > On 2021-Mar-07, Magnus Hagander wrote: > > > > > > >

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-09 Thread James Coleman
On Tue, Mar 9, 2021 at 9:17 AM Alvaro Herrera wrote: > > On 2021-Mar-09, James Coleman wrote: > > > Yes, I think they both agreed on the "DETAIL: Hot standby mode is > > disabled." message, but that alternative meant not needing to add any > > new signals

Re: enable_incremental_sort changes query behavior

2020-10-30 Thread James Coleman
On Thu, Oct 29, 2020 at 6:06 PM Tomas Vondra wrote: > > On Tue, Oct 06, 2020 at 09:37:31AM -0400, James Coleman wrote: > >On Tue, Oct 6, 2020 at 9:28 AM Jaime Casanova > > wrote: > >> Can you please create an entry in the commitfest for this one so we > >> d

Re: enable_incremental_sort changes query behavior

2020-10-30 Thread James Coleman
On Fri, Oct 30, 2020 at 5:03 PM Tomas Vondra wrote: > > On Fri, Oct 30, 2020 at 01:26:08PM -0400, James Coleman wrote: > >On Thu, Oct 29, 2020 at 6:06 PM Tomas Vondra > > wrote: > >> > >> On Tue, Oct 06, 2020 at 09:37:31AM -0400, James Coleman wrote: > &

Re: Use of "long" in incremental sort code

2020-11-04 Thread James Coleman
>FWIW while looking at what the other places calling LogicalTapeSetBlocks > >do, and I noticed this: > > > >uint64 disk_used = LogicalTapeSetBlocks(...); > > > >in the disk-based hashagg patch. So that's a third data type ... > > > > IMHO this should simply switch the current int64 variable to long, as it > was before. Not sure about about the hashagg uint64 variable. Is there anything that actually limits tape code to using at most 4GB on 32-bit systems? James

Re: enable_incremental_sort changes query behavior

2020-11-17 Thread James Coleman
, as I understand it, we did do sorts below gather nodes before, just in fewer places, which meant that "in theory" the code was already broken (or at least not complete) for parallel queries. James

[PATCH 1/1] Fix compilation on mac with Xcode >= 11.4.

2020-11-19 Thread James Hilliard
It would appear weak symbol linking is not handled properly without 'isysroot' parameter passed to linker. Fixes: Undefined symbols for architecture x86_64: "___darwin_check_fd_set_overflow", referenced from: _ClientAuthentication in auth.o _pgstat_init in pgstat.o _ServerLoop

Re: [PATCH 1/1] Fix compilation on mac with Xcode >= 11.4.

2020-11-19 Thread James Hilliard
On Thu, Nov 19, 2020 at 5:40 PM Tom Lane wrote: > > James Hilliard writes: > > It would appear weak symbol linking is not handled properly without > > 'isysroot' parameter passed to linker. > > Nobody else has reported this problem, so maybe you should tell u

Re: [PATCH 1/1] Fix compilation on mac with Xcode >= 11.4.

2020-11-19 Thread James Hilliard
Xcode sysroot, so I think this patch is still correct in that case as it forces the use of the Xcode sysroot for linking which should be what we want as we want it to match sysroot passed to the compiler. On Thu, Nov 19, 2020 at 5:45 PM James Hilliard wrote: > > On Thu, Nov 19, 2020 at 5

Re: [PATCH 1/1] Fix compilation on mac with Xcode >= 11.4.

2020-11-19 Thread James Hilliard
On Thu, Nov 19, 2020 at 6:04 PM Tom Lane wrote: > > James Hilliard writes: > > Hmm, maybe it's a more recent issue then, I took the version number > > from the qt patch assuming it was the same issue, I hit it trying to build > > master on Xcode 12.2 Build versi

Re: [PATCH 1/1] Fix compilation on mac with Xcode >= 11.4.

2020-11-19 Thread James Hilliard
On Thu, Nov 19, 2020 at 7:20 PM Tom Lane wrote: > > James Hilliard writes: > > On Thu, Nov 19, 2020 at 6:04 PM Tom Lane wrote: > >> The cases we've got in the buildfarm are Xcode 12.0 on Catalina (10.15.7) > >> and Xcode 12.2 on Big Sur (11.0.1 ...

Re: [PATCH 1/1] Fix compilation on mac with Xcode >= 11.4.

2020-11-19 Thread James Hilliard
On Thu, Nov 19, 2020 at 7:48 PM Tom Lane wrote: > > I wrote: > > However ... it then occurred to me to blow away my ccache and accache, > > and after rebuilding from scratch, everything's fine. So, are you > > using ccache? > > Oh, scratch that, I fat-fingered the experiment somehow. The issue >

Re: [PATCH 1/1] Fix compilation on mac with Xcode >= 11.4.

2020-11-19 Thread James Hilliard
On Thu, Nov 19, 2020 at 9:02 PM Tom Lane wrote: > > James Hilliard writes: > > On Thu, Nov 19, 2020 at 7:48 PM Tom Lane wrote: > >> Oh, scratch that, I fat-fingered the experiment somehow. The issue > >> is still there. Still, I'm hesitant to apply the fix y

Re: enable_incremental_sort changes query behavior

2020-11-20 Thread James Coleman
On Tue, Nov 17, 2020 at 11:20 AM Tomas Vondra wrote: > > > > On 11/17/20 3:03 PM, James Coleman wrote: > > On Mon, Nov 16, 2020 at 11:23 PM Tomas Vondra > > wrote: > >> > >> Hmm, I missed that other thread. That indeed seems like a bug in the > >

Re: enable_incremental_sort changes query behavior

2020-11-20 Thread James Coleman
ro case in [1]'s discussion about parallel safety here) is a counterexample. Specifically we have a plan like: Merge Right Join -> Unique -> Gather Merge -> Sort -> Nested Loop The pathtarget of the nested loop contains non-var expressions (in this case

Re: enable_incremental_sort changes query behavior

2020-11-20 Thread James Coleman
Thanks much for the detailed followup; this is super helpful. On Fri, Nov 20, 2020 at 2:57 PM Robert Haas wrote: > > On Fri, Nov 20, 2020 at 1:51 PM James Coleman wrote: > > > This isn't a counterexample, because there's no join tree here -- or, > > > well, t

Why does create_gather_merge_plan need make_sort?

2020-11-20 Thread James Coleman
appened to know what case is being handled by the make_sort call. Removing it doesn't seem to break any tests. Thanks, James

Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-11-20 Thread James Coleman
#x27;s safe (though I haven't thought about it a ton) because I believe those are sort keys we're going to send to the foreign server anyway, as opposed to sorting ourselves (but I haven't verified that that's the case and that we never create sort nodes there). Jam

[PATCH 1/1] Initial mach based shared memory support.

2020-11-22 Thread James Hilliard
OSX implements sysv shmem support via a mach wrapper, however the mach sysv shmem wrapper has some severe restrictions that prevent us from allocating enough memory segments in some cases. These limits appear to be due to the way the wrapper itself is implemented and not mach. For example when ru

Re: Why does create_gather_merge_plan need make_sort?

2020-11-23 Thread James Coleman
On Sun, Nov 22, 2020 at 5:07 PM Tomas Vondra wrote: > > On 11/22/20 10:31 PM, Tom Lane wrote: > > Tomas Vondra writes: > >> On 11/20/20 11:24 PM, James Coleman wrote: > >>> While looking at another issue I noticed that create_gather_merge_plan > >

Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-11-23 Thread James Coleman
On Sun, Nov 22, 2020 at 4:59 PM Tomas Vondra wrote: > > On 11/21/20 2:55 AM, James Coleman wrote: > > Over on the -bugs list we had a report [1] of a seg fault with > > incremental sort. The short of the investigation there was that a > > subplan wasn't being serial

Re: enable_incremental_sort changes query behavior

2020-11-23 Thread James Coleman
On Mon, Nov 23, 2020 at 2:24 PM Tom Lane wrote: > > James Coleman writes: > > But I think that still leaves something missing: after all, > > prepare_sort_from_pathkeys does know how to insert new target list > > entries, so something either there (or in the caller/how i

Re: enable_incremental_sort changes query behavior

2020-11-25 Thread James Coleman
On Tue, Nov 24, 2020 at 2:31 PM Tom Lane wrote: > > James Coleman writes: > > On Mon, Nov 23, 2020 at 2:24 PM Tom Lane wrote: > >> 1. James was wondering, far upthread, why we would do projections > >> pre-sort or post-sort. I think the answers can be foun

Re: enable_incremental_sort changes query behavior

2020-11-25 Thread James Coleman
On Wed, Nov 25, 2020 at 2:53 PM James Coleman wrote: > > On Tue, Nov 24, 2020 at 2:31 PM Tom Lane wrote: > > > > James Coleman writes: > > > On Mon, Nov 23, 2020 at 2:24 PM Tom Lane wrote: > > >> 1. James was wondering, far upthread, why we would do proje

Re: Fix generate_useful_gather_paths for parallel unsafe pathkeys

2020-11-29 Thread James Coleman
On Mon, Nov 23, 2020 at 8:35 AM James Coleman wrote: > > On Sun, Nov 22, 2020 at 4:59 PM Tomas Vondra > wrote: > > ... > > Thanks for the patches, I'll take a closer look next week. The 0002 > > patch is clearly something we need to backpatch, not sure about 00

Re: Why does create_gather_merge_plan need make_sort?

2020-11-29 Thread James Coleman
On Mon, Nov 23, 2020 at 8:19 AM James Coleman wrote: > > On Sun, Nov 22, 2020 at 5:07 PM Tomas Vondra > wrote: > > > > On 11/22/20 10:31 PM, Tom Lane wrote: > > > Tomas Vondra writes: > > >> On 11/20/20 11:24 PM, James Coleman wrote: > > &

Re: [DOC] Document concurrent index builds waiting on each other

2021-01-13 Thread James Coleman
nk one more para in the notes is good. But shouldn't we still clarify the issue is specific to CONCURRENTLY? Also that it's not just the table being indexed seems fairly significant. How about something like: --- Like any long-running transaction, REINDEX CONCURRENTLY can affect which tuples can be removed by concurrent VACUUM on any table. --- James

Re: [DOC] Document concurrent index builds waiting on each other

2021-01-13 Thread James Coleman
On Wed, Jan 13, 2021 at 12:33 PM Alvaro Herrera wrote: > > On 2021-Jan-13, James Coleman wrote: > > > On Wed, Jan 13, 2021 at 12:58 AM Michael Paquier > > wrote: > > > > > > On Tue, Jan 12, 2021 at 04:51:39PM -0300, Alvaro Herrera wrote: > > > &g

Re: [DOC] Document concurrent index builds waiting on each other

2021-01-13 Thread James Coleman
On Wed, Jan 13, 2021 at 4:05 PM Alvaro Herrera wrote: > > On 2021-Jan-13, James Coleman wrote: > > > On Wed, Jan 13, 2021 at 12:33 PM Alvaro Herrera > > wrote: > > > > This is true. So I propose > > > > > > Like any long-running transactio

Re: [DOC] Document concurrent index builds waiting on each other

2021-01-13 Thread James Coleman
On Wed, Jan 13, 2021 at 4:29 PM Tom Lane wrote: > > Alvaro Herrera writes: > > On 2021-Jan-13, James Coleman wrote: > >>>> This is true. So I propose > >>>> Like any long-running transaction, REINDEX can > >>>> affect which tuples can be

Re: [DOC] Document concurrent index builds waiting on each other

2021-01-13 Thread James Coleman
On Wed, Jan 13, 2021 at 5:00 PM Tom Lane wrote: > > James Coleman writes: > > On Wed, Jan 13, 2021 at 4:29 PM Tom Lane wrote: > >> but the antecedent of "a table" is a bit unclear; people might > >> wonder if it means the table being reindexed. > >

Re: [PATCH 1/1] Initial mach based shared memory support.

2021-01-18 Thread James Hilliard
On Mon, Jan 18, 2021 at 5:29 PM Tom Lane wrote: > > James Hilliard writes: > > OSX implements sysv shmem support via a mach wrapper, however the mach > > sysv shmem wrapper has some severe restrictions that prevent us from > > allocating enough memory segments in some cas

Re: [PATCH 1/1] Initial mach based shared memory support.

2021-01-18 Thread James Hilliard
On Mon, Jan 18, 2021 at 5:29 PM Tom Lane wrote: > > James Hilliard writes: > > OSX implements sysv shmem support via a mach wrapper, however the mach > > sysv shmem wrapper has some severe restrictions that prevent us from > > allocating enough memory segments in some cas

[PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
Fixes: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot /Applications/Xcode.app/Contents/De

Re: [PATCH 1/1] Initial mach based shared memory support.

2021-01-19 Thread James Hilliard
On Mon, Jan 18, 2021 at 11:12 PM Tom Lane wrote: > > James Hilliard writes: > > from my understanding due to mach semantics a number of the sanity checks > > we are doing for sysv shmem are probably unnecessary when using mach > > API's due to the mach port

Re: pg_preadv() and pg_pwritev()

2021-01-19 Thread James Hilliard
On Thu, Jan 14, 2021 at 6:45 PM Tom Lane wrote: > > Sergey Shinderuk writes: > > On 15.01.2021 01:13, Tom Lane wrote: > >> Also, after re-reading [1] I am not at all excited about trying to > >> remove the -isysroot switches from our *FLAGS. What I propose to do > >> is keep that, but improve ou

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 8:27 AM Tom Lane wrote: > > James Hilliard writes: > > Fixes: > > gcc -Wall -Wmissing-prototypes -Wpointer-arith > > -Wdeclaration-after-statement -Werror=vla -Wendif-labels > > -Wmissing-format-attribute -Wformat-security -fno-strict-ali

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 8:57 AM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 8:27 AM Tom Lane wrote: > >> We already dealt with that by not selecting an SDK newer than the > >> underlying OS (see 4823621db). > > > Tried that, d

[PATCH v2 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
Fixes: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot /Applications/Xcode.app/Contents/De

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 10:17 AM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 8:57 AM Tom Lane wrote: > >> It worked for me and for Sergey, so we need to figure out what's different > >> about your setup. What do you get from &quo

Re: [PATCH v2 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 10:29 AM Tom Lane wrote: > > James Hilliard writes: > > Fixes: > > fd.c:3661:10: warning: 'pwritev' is only available on macOS 11.0 or newer > > [-Wunguarded-availability-new] > > It's still missing preadv, and it stil

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 1:54 PM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 10:17 AM Tom Lane wrote: > >> Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that > >> is, it *does* give us a symlink to a 10.15 SD

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-19 Thread James Hilliard
On Tue, Jan 19, 2021 at 3:47 PM James Hilliard wrote: > > On Tue, Jan 19, 2021 at 1:54 PM Tom Lane wrote: > > > > James Hilliard writes: > > > On Tue, Jan 19, 2021 at 10:17 AM Tom Lane wrote: > > >> Ah, got it. So "xcrun --show-sdk-path" t

Re: Discarding DISCARD ALL

2021-01-20 Thread James Coleman
w setting session level GUCs (i.e., enforce `SET LOCAL` transaction level usage instead) would remove a large chunk of our need to set server_reset_query_always=1 (and more interestingly it'd highlight when broken code gets pushed). But even with that, I see some value in the proposed setting since there is additional session state beyond GUCs. James

Re: Discarding DISCARD ALL

2021-01-20 Thread James Coleman
On Wed, Jan 20, 2021 at 9:58 AM Simon Riggs wrote: > > On Wed, 20 Jan 2021 at 14:21, James Coleman wrote: > > > An alternative approach that occurred to me while typing this reply: a > > setting in Postgres that would disallow setting session level GUCs > > (i.e., enfo

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-20 Thread James Hilliard
On Tue, Jan 19, 2021 at 6:37 PM Tom Lane wrote: > > James Hilliard writes: > > Actually, this looks path looks wrong in general, the value for > > "xcrun --sdk macosx --show-sdk-path" should take precedence over > > "xcrun --show-sdk-path" as the latt

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-20 Thread James Hilliard
On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 6:37 PM Tom Lane wrote: > >> I've found no direct means to control the > >> SDK path at all, but so far it appears that "xcrun --show-sdk-path&

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-21 Thread James Hilliard
On Thu, Jan 21, 2021 at 11:38 AM Tom Lane wrote: > > James Hilliard writes: > > On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: > >> I'm not sure that the case of not having the "command line tools" > >> installed is interesting for our purposes.

[PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-01-22 Thread James Hilliard
Fixes: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot /Applications/Xcode.app/Contents/De

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-22 Thread James Hilliard
On Thu, Jan 21, 2021 at 11:38 AM Tom Lane wrote: > > James Hilliard writes: > > On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: > >> I'm not sure that the case of not having the "command line tools" > >> installed is interesting for our purposes.

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-23 Thread James Hilliard
On Sat, Jan 23, 2021 at 1:27 AM Sergey Shinderuk wrote: > > On 23.01.2021 08:02, Sergey Shinderuk wrote: > >> On the whole it looks like we should recommend installing the CLT > >> and not bothering with Xcode, which is about 10X the size: > >> > >> $ du -hs /Library/Developer/CommandLineTools > >

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-01-30 Thread James Hilliard
On Fri, Jan 22, 2021 at 12:32 PM James Hilliard wrote: > > Fixes: > gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement > -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security > -fno-strict-aliasing -fwrapv -Wno-unused-command-line-arg

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-02-05 Thread James Hilliard
On Fri, Jan 22, 2021 at 1:53 PM James Hilliard wrote: > > On Thu, Jan 21, 2021 at 11:38 AM Tom Lane wrote: > > > > James Hilliard writes: > > > On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: > > >> I'm not sure that the case of not having

Re: Consider parallel for lateral subqueries with limit

2022-01-04 Thread James Coleman
when it's a lateral reference. That limit/scan (if lateral) has to be being executed per tuple in the outer scan, right? And if it's a unique execution per tuple, then consistency across tuples (that are in different workers) can't be a concern. Is there a scenario I'm missing where lateral can currently be executed in that way in that structure (or a different one)? Thanks, James Coleman

Re: Consider parallel for lateral subqueries with limit

2022-01-13 Thread James Coleman
On Tue, Jan 4, 2022 at 9:59 PM James Coleman wrote: > > On Tue, Jan 4, 2022 at 5:31 PM Tom Lane wrote: > > > > Greg Nancarrow writes: > > > The patch LGTM. > > > I have set the status to "Ready for Committer". > > > > I don't really

Re: Parallelize correlated subqueries that execute within each worker

2022-01-14 Thread James Coleman
On Fri, Dec 3, 2021 at 2:35 AM Michael Paquier wrote: > > On Mon, Nov 15, 2021 at 10:01:37AM -0500, Robert Haas wrote: > > On Wed, Nov 3, 2021 at 1:34 PM James Coleman wrote: > >> As I understand the current code, parallel plans are largely chosen > >> based not

Re: Parallelize correlated subqueries that execute within each worker

2022-01-14 Thread James Coleman
On Mon, Nov 15, 2021 at 10:01 AM Robert Haas wrote: > > On Wed, Nov 3, 2021 at 1:34 PM James Coleman wrote: > > As I understand the current code, parallel plans are largely chosen > > based not on where it's safe to insert a Gather node but rather by > > determinin

Add last commit LSN to pg_last_committed_xact()

2022-01-14 Thread James Coleman
on't write it out to disk). That the downside of making this feature dependent on "track_commit_timestamps = on", but that seems reasonable: 1. Getting the xid of the last commit is similarly dependent on commit timestamps infrastructure. 2. It's a simple place to hook into

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-19 Thread James Coleman
On Tue, Mar 9, 2021 at 9:27 AM Fujii Masao wrote: > > > > On 2021/03/09 23:19, James Coleman wrote: > > On Tue, Mar 9, 2021 at 9:17 AM Alvaro Herrera > > wrote: > >> > >> On 2021-Mar-09, James Coleman wrote: > >> > >>> Yes

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-03-19 Thread James Coleman
On Fri, Sep 11, 2020 at 5:11 PM James Coleman wrote: > > On Tue, Sep 8, 2020 at 4:37 PM Heikki Linnakangas wrote: > > > > On 08/09/2020 22:25, James Coleman wrote: > > > On Wed, Aug 19, 2020 at 3:16 AM Heikki Linnakangas > > > wrote: > > >> >

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-22 Thread James Coleman
On Mon, Mar 22, 2021 at 1:49 PM Fujii Masao wrote: > > > > On 2021/03/19 23:35, James Coleman wrote: > > Here's an updated patch; I think I've gotten what Alvaro suggested. > > Thanks for updating the patch! But I was thinking that our consensus is > somethi

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-22 Thread James Coleman
On Mon, Mar 22, 2021 at 2:52 PM Fujii Masao wrote: > > > > On 2021/03/23 3:25, James Coleman wrote: > > On Mon, Mar 22, 2021 at 1:49 PM Fujii Masao > > wrote: > >> > >> > >> > >> On 2021/03/19 23:35, James Coleman wrote: > >>

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread James Coleman
On Tue, Mar 23, 2021 at 1:46 AM Fujii Masao wrote: > > > > On 2021/03/23 3:59, James Coleman wrote: > > Are you saying we should only change the message for a single case: > > the case where we'd otherwise allow connections but EnableHotStandby > > is false

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-23 Thread James Coleman
say "... is not accepting connections *yet*". > > +1, but I think "... is not yet accepting connections" is slightly > better style. All right, see attached v8. James Coleman v8-0001-Improve-standby-connection-denied-error-message.patch Description: Binary data

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-24 Thread James Coleman
results. If somebody saw this during a normal (non-crash) startup, > >>> they'd be justifiably alarmed. > >> > >> Yes, so logging "the database system is starting up" seems enough to me. > > > > No objection. > > Thanks! So I changed the m

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-03-29 Thread James Hilliard
Should it work if I just attach it to the thread like this? On Mon, Mar 29, 2021 at 7:52 AM David Steele wrote: > > Hi James, > > On 1/31/21 1:59 AM, James Hilliard wrote: > > On Fri, Jan 22, 2021 at 12:32 PM James Hilliard > > wrote: > >> > >>

Fix detection of preadv/pwritev support for OSX.

2021-03-29 Thread James Hilliard
This fixes an issue where preadv and pwritev detection does not properly respect the OSX deployment target version symbol availability. James v3-0001-Fix-detection-of-preadv-pwritev-support-for-OSX.patch Description: Binary data

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-03-29 Thread James Hilliard
On Mon, Mar 29, 2021 at 4:10 PM Thomas Munro wrote: > > On Tue, Mar 30, 2021 at 6:37 AM James Hilliard > wrote: > > Should it work if I just attach it to the thread like this? > > Yes. It automatically tries patches that are attached to threads that > are registered on c

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-03-29 Thread James Hilliard
On Mon, Mar 29, 2021 at 11:58 PM Tom Lane wrote: > > Thomas Munro writes: > > I'll move it when committing. I'll let this patch sit for another day > > to see if any other objections show up. > > FWIW, I remain fairly strongly against this, precisely because of the > point that it requires us to

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-03-30 Thread James Hilliard
On Tue, Mar 30, 2021 at 6:43 PM Thomas Munro wrote: > > On Tue, Mar 30, 2021 at 7:39 PM James Hilliard > wrote: > > On Mon, Mar 29, 2021 at 11:58 PM Tom Lane wrote: > > > We haven't claimed in the past to support MACOSX_DEPLOYMENT_TARGET, > > > and I

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-03-30 Thread James Hilliard
On Tue, Mar 30, 2021 at 7:51 PM Tom Lane wrote: > > Thomas Munro writes: > > Personally I'm mostly concerned about making it easy for new > > contributors to get a working dev system going on a super common > > platform without dealing with hard-to-diagnose errors, than people who > > actually wa

Re: [EXTERNAL] Any objection to documenting pg_sequence_last_value()?

2021-04-06 Thread James Coleman
elp document this function if there are no objections. > > Best, > Hanefi > > -Original Message- > From: James Coleman > Sent: 6 Ağustos 2020 Perşembe 16:14 > To: pgsql-hackers > Subject: [EXTERNAL] Any objection to documenting pg_sequence_last_value()? > > T

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-06 Thread James Coleman
On Mon, Apr 5, 2021 at 11:58 PM David Rowley wrote: > > On Sat, 20 Mar 2021 at 09:41, James Coleman wrote: > > I've attached a cleaned up patch. Last CF it was listed in is > > https://commitfest.postgresql.org/29/2542/ -- what's the appropriate > > step

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread James Coleman
't have permissions?) to move it into the current CF. How does one go about doing that? Here's the entry: https://commitfest.postgresql.org/29/2542/ Thanks, James

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread James Coleman
On Thu, Apr 8, 2021 at 1:04 PM Alvaro Herrera wrote: > > On 2021-Apr-08, James Coleman wrote: > > > I assume proper procedure for the CF entry is to move it into the > > current CF and then mark it as committed, however I don't know how (or > > don't have perm

Processing btree walks as a batch to parallelize IO

2021-04-09 Thread James Coleman
e've been testing I believe such an approach could plausibly improve table insert (and update) performance by multiple hundreds of percent. I don't have any code at the moment to show here, but I wanted to get the idea out there to see if there were any immediate reactions or other thoughts on the topic. Thoughts? James

Re: Nicer error when connecting to standby with hot_standby=off

2021-04-09 Thread James Coleman
On Wed, Mar 24, 2021 at 9:43 PM Fujii Masao wrote: > > > > On 2021/03/24 22:06, James Coleman wrote: > > That looks good to me. Thanks for working on this. > > Thanks! I pushed the patch. > > Regards, > > -- > Fujii Masao > Advanced Computing Te

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-12 Thread James Coleman
e patch. But also I didn't think a whole lot about the NOT IN case at all -- and there's no mention of such that I see in this thread or the precursor thread. It's pretty obvious that it wasn't part of my immediate need, but obviously it'd be nice to have the consistency. All that to say this: my vote would be to put it into PG15 also. James

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-13 Thread James Coleman
On Mon, Apr 12, 2021 at 10:07 PM James Coleman wrote: > > On Mon, Apr 12, 2021 at 7:49 PM David Rowley wrote: > > > > On Tue, 13 Apr 2021 at 11:42, Tom Lane wrote: > > > > > > David Rowley writes: > > > > I realised when working on something

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread James Coleman
n (argc=5, argv=0x1f4e8e0) at > postmaster.c:1400 > #25 0x007823f3 in main (argc=5, argv=0x1f4e8e0) at main.c:210 > > This appears to be an issue with d2d8a229b (Incremental Sort), so I will add > at: https://wiki.postgresql.org/wiki/PostgreSQL_13_Open_Items Is that assumption largely based on the incremental sort patch refactoring tuplesort.c a bit? I haven't looked at it much at all, but I'm wondering if the issue could also be related to the btree duplicates changes in 13 given that we're looking at comparetup_index_btree and the datums are equal. James

Re: Threading in BGWorkers (!)

2020-07-28 Thread James Sewell
> We need some kind of concrete plan for what is a > usable amount of functionality and what has to be done to get it. > This is exactly the type of discussion I'm after. I'll start. A usable amount of functionality would be the ability to start threads from: - a background worker These cas

  1   2   3   4   5   6   7   >