Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-20 Thread Etsuro Fujita
mitted in parallel. (To say that this option is not for a specific foreign server, I added to the documentation “This option can only be specified for foreign servers”.) > "in a local (sub)transaction" part seems not to be necessary. And I thought adding it would make clear which remote transactions are committed in parallel. But maybe I’m missing something, so could you elaborate a bit more on these? Thanks for reviewing! Best regards, Etsuro Fujita

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-20 Thread Etsuro Fujita
der redefine this number a > macro somewhere? Yeah, I think that is a good idea. I’ll do so in the next version of the parallel-abort patch (#0003) if no objections. Best regards, Etsuro Fujita

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-22 Thread Etsuro Fujita
On Tue, Feb 22, 2022 at 1:03 AM Fujii Masao wrote: > On 2022/02/21 14:45, Etsuro Fujita wrote: > > On Fri, Feb 18, 2022 at 1:46 AM Fujii Masao > > wrote: > >> I reviewed 0001 patch. It looks good to me except the following minor > >> things. If these are addr

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-23 Thread Etsuro Fujita
On Wed, Feb 23, 2022 at 3:30 PM Etsuro Fujita wrote: > Attached is an updated patch. > Barring objections, I’ll commit > the patch. I have committed the patch. I think the 0003 patch needs rebase. I'll update the patch. Best regards, Etsuro Fujita

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-28 Thread Etsuro Fujita
On Thu, Feb 24, 2022 at 2:49 PM Etsuro Fujita wrote: > I think the 0003 patch needs rebase. > I'll update the patch. Here is an updated version. I added to the 0003 patch a macro for defining the milliseconds to wait, as proposed by David upthread. Best regards, Etsuro Fujit

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-03-05 Thread Etsuro Fujita
On Mon, Feb 28, 2022 at 6:53 PM Etsuro Fujita wrote: > Here is an updated version. I added to the 0003 patch a macro for > defining the milliseconds to wait, as proposed by David upthread. I modified the 0003 patch further: 1) I added to pgfdw_cancel_query_end/pgfdw_exec_cleanup_query_e

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-03-13 Thread Etsuro Fujita
2 = 4.1572 ms > > after the patches with OPTIONS (ADD parallel_abort 'true'): > > abort.1 = 1.7136 ms > > abort.2 = 2.5833 ms > > Overall, it has about 32 ~ 37 % improvement in my testing environment. I think that is a great improvement. Thanks for testing! Best regards, Etsuro Fujita

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-03-13 Thread Etsuro Fujita
Hi Alexander, On Wed, Sep 15, 2021 at 3:40 PM Alexander Pyhalov wrote: > Etsuro Fujita писал 2021-08-30 12:52: > > To allow async execution in a bit more cases, I modified the patch a > > bit further: a ProjectionPath put directly above an async-capable > > ForeignPath wou

Re: Commitfest Update

2022-03-18 Thread Etsuro Fujita
On Fri, Mar 18, 2022 at 3:08 AM Greg Stark wrote: > In case anyone's looking for inspiration... Here are the list of > patches marked Ready for Committer: > * Fast COPY FROM command for the foreign tables I have (re-)started reviewing this patch. Best regards, Etsuro Fujita

Re: Fast COPY FROM based on batch insert

2022-03-21 Thread Etsuro Fujita
ish my review, but I’ll mark this as “Waiting on Author”. My apologies for the long long delay. Best regards, Etsuro Fujita

Re: Fast COPY FROM based on batch insert

2022-03-21 Thread Etsuro Fujita
PY approaches. > > Patch doesn't apply and likely hasn't for a while... Actually, it has bit-rotted due to the recent fix for cross-partition updates (i.e., commit ba9a7e392). Thanks! Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2020-12-12 Thread Etsuro Fujita
On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi wrote: > At Tue, 17 Nov 2020 18:56:02 +0900, Etsuro Fujita > wrote in > > * In Robert's patch [1] (and Horiguchi-san's, which was created based > > on Robert's), ExecAppend() was modified to retrieve tuples from

Re: Asynchronous Append on postgres_fdw nodes.

2020-12-12 Thread Etsuro Fujita
. > That being said, I think we should unify the > code except the differences between async and sync. For example, if > the fetch_more_data_begin() needs to be called only for async > fetching, the cursor should be created before calling the function, in > the code path common with sy

Re: Asynchronous Append on postgres_fdw nodes.

2020-12-19 Thread Etsuro Fujita
On Mon, Dec 14, 2020 at 4:01 PM Kyotaro Horiguchi wrote: > At Sat, 12 Dec 2020 18:25:57 +0900, Etsuro Fujita > wrote in > > On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi > > wrote: > > > The reason for > > > the early fetching is letting fdw send the nex

Re: Asynchronous Append on postgres_fdw nodes.

2020-12-19 Thread Etsuro Fujita
On Mon, Dec 14, 2020 at 5:56 PM Kyotaro Horiguchi wrote: > At Sat, 12 Dec 2020 19:06:51 +0900, Etsuro Fujita > wrote in > > On Fri, Nov 20, 2020 at 8:16 PM Kyotaro Horiguchi > > wrote: > > > + /* wait or poll async events */ > > > +

Re: Asynchronous Append on postgres_fdw nodes.

2020-12-20 Thread Etsuro Fujita
ry to the remote, 2) it would then process the local partition until the end, 3) it would then wait/poll the async event, and 4) it would finally process the remote partition when the event occurs. Sorry for the delay. Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2020-12-20 Thread Etsuro Fujita
On Thu, Dec 10, 2020 at 3:38 PM Andrey V. Lepikhov wrote: > On 11/17/20 2:56 PM, Etsuro Fujita wrote: > > On Mon, Oct 5, 2020 at 3:35 PM Etsuro Fujita > > wrote: > > Comments welcome! The attached is still WIP and maybe I'm missing > > something, though. > I

Re: Asynchronous Append on postgres_fdw nodes.

2020-12-31 Thread Etsuro Fujita
On Sat, Dec 19, 2020 at 5:55 PM Etsuro Fujita wrote: > On Mon, Dec 14, 2020 at 4:01 PM Kyotaro Horiguchi > wrote: > > At Sat, 12 Dec 2020 18:25:57 +0900, Etsuro Fujita > > wrote in > > > On Fri, Nov 20, 2020 at 3:51 PM Kyotaro Horiguchi > > > wrote: >

Re: Asynchronous Append on postgres_fdw nodes.

2021-01-01 Thread Etsuro Fujita
On Thu, Dec 31, 2020 at 7:15 PM Etsuro Fujita wrote: > * I tweaked comments a bit to address your comments. I forgot to update some comments. :-( Attached is a new version of the patch updating comments further. I did a bit of cleanup for the postgres_fdw part as well. Best regards, Ets

Re: Asynchronous Append on postgres_fdw nodes.

2021-01-02 Thread Etsuro Fujita
On Sun, Dec 20, 2020 at 5:15 PM Etsuro Fujita wrote: > On Thu, Nov 26, 2020 at 10:28 AM movead...@highgo.ca > wrote: > > Issue one: > > Get a Assert error at 'Assert(bms_is_member(i, node->as_needrequest));' in > > ExecAppendAsyncRequest() function when I u

Re: Asynchronous Append on postgres_fdw nodes.

2021-02-14 Thread Etsuro Fujita
ns that use the same connection, they won’t be parallelized at all, and the overhead of async execution may cause a performance degradation. So the patch disables async execution of them in that case using the above code bit. Thanks for the review! Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2021-02-17 Thread Etsuro Fujita
On Wed, Feb 10, 2021 at 9:31 PM Etsuro Fujita wrote: > Please find attached an updated patch. I noticed that this doesn’t work for cases where ForeignScans are executed inside functions, and I don’t have any simple solution for that. So I’m getting back to what Horiguchi-san proposed

Re: Asynchronous Append on postgres_fdw nodes.

2021-02-19 Thread Etsuro Fujita
On Thu, Feb 18, 2021 at 3:16 PM Kyotaro Horiguchi wrote: > At Thu, 18 Feb 2021 11:51:59 +0900, Etsuro Fujita > wrote in > > I noticed that this doesn’t work for cases where ForeignScans are > > executed inside functions, and I don’t have any simple solution for > > A

Re: Asynchronous Append on postgres_fdw nodes.

2021-03-01 Thread Etsuro Fujita
On Sat, Feb 20, 2021 at 3:35 PM Etsuro Fujita wrote: > So I’ll update the patch using your patch without the > postgresIterateForeignScan() change. Here is an updated version of the patch. Based on your idea of completing an in-progress command (if any) before sending a new command

Re: Asynchronous Append on postgres_fdw nodes.

2021-03-03 Thread Etsuro Fujita
On Mon, Mar 1, 2021 at 5:56 PM Etsuro Fujita wrote: > Here is an updated version of the patch. Another thing I'm concerned about in the postgres_fdw part is the case where all/many postgres_fdw ForeignScans of an Append use the same connection, because in that case those ForeignS

Re: Asynchronous Append on postgres_fdw nodes.

2021-03-07 Thread Etsuro Fujita
On Thu, Mar 4, 2021 at 1:00 PM Etsuro Fujita wrote: > To avoid that, how about 1) adding the > table/server options to postgres_fdw that allow/disallow async > execution, and 2) setting them to false by default? There seems to be no objections, so I went ahead and added the table/serv

Re: Asynchronous Append on postgres_fdw nodes.

2021-03-07 Thread Etsuro Fujita
On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita wrote: > * I haven't yet added some planner/resowner changes from Horiguchi-san's > patch. The patch in [1] allocates, populates, frees a wait event set every time when doing ExecAppendAsyncEventWait(), so it wouldn’t leak wait event

Re: Asynchronous Append on postgres_fdw nodes.

2020-11-12 Thread Etsuro Fujita
ould impact performance *negatively* in some cases as mentioned in [1]. So I feel inclined to just disable this feature in problematic cases including the above one in the first cut. Even with such a limitation, I think it would be useful, because it would cover typical use cases such as pa

Re: Asynchronous Append on postgres_fdw nodes.

2020-11-12 Thread Etsuro Fujita
h in attachment tries to improve this situation. Seems like a good idea. Will look at the patch in detail. Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2020-11-17 Thread Etsuro Fujita
On Mon, Oct 5, 2020 at 3:35 PM Etsuro Fujita wrote: > Yes, if there are no objections from you or Thomas or Robert or anyone > else, I'll update Robert's patch as such. Here is a new version of the patch (as promised in the developer unconference in PostgresConf.CN & PGCo

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-11-23 Thread Etsuro Fujita
tch that needed to be corrected. > It is wait for actions from committers side, i think. I'm planning to review this patch. I think it would be better for another pair of eyes to take a look at it, though. Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2021-01-17 Thread Etsuro Fujita
outer/inner sides didn’t match, we could iterate calling *ExecAsyncRequest()* for the inner side until a matched tuple from it is found. If the inner side wasn’t able to return a tuple immediately, 1) it would return request_complete=false to Join-2 using ExecAsyncResponse(), and 2) we could wait for a file descriptor for the inner side to become ready (while processing other part of the Append tree), and 3) when the file descriptor becomes ready, recursive ExecAsyncNotify() calls would restart the Join-2 processing in a push-based manner as explained above. Best regards, Etsuro Fujita [1] https://www.postgresql.org/message-id/CAPmGK14xrGe%2BXks7%2BfVLBoUUbKwcDkT9km1oFXhdY%2BFFhbMjUg%40mail.gmail.com

Re: Asynchronous Append on postgres_fdw nodes.

2021-01-31 Thread Etsuro Fujita
On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita wrote: > * I haven't yet done anything about the issue on postgres_fdw's > handling of concurrent data fetches by multiple ForeignScan nodes > (below *different* Append nodes in the query) using the same > connection discussed i

Re: Asynchronous Append on postgres_fdw nodes.

2021-02-04 Thread Etsuro Fujita
On Mon, Feb 1, 2021 at 12:06 PM Etsuro Fujita wrote: > On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita wrote: > > * I haven't yet done anything about the issue on postgres_fdw's > > handling of concurrent data fetches by multiple ForeignScan nodes > > (below *diffe

Re: Asynchronous Append on postgres_fdw nodes.

2021-02-10 Thread Etsuro Fujita
On Thu, Feb 4, 2021 at 7:21 PM Etsuro Fujita wrote: > On Mon, Feb 1, 2021 at 12:06 PM Etsuro Fujita wrote: > > Rather than doing so, I'd like to propose to allow > > FDWs to disable async execution of them in problematic cases by > > themselves during executor startup

Re: Asynchronous Append on postgres_fdw nodes.

2021-02-10 Thread Etsuro Fujita
On Wed, Feb 10, 2021 at 7:31 PM Etsuro Fujita wrote: > Attached is an updated version of the patch. Sorry for the delay. I noticed that I forgot to add new files. :-(. Please find attached an updated patch. Best regards, Etsuro Fujita async-wip-2021-02-10-v2.patch Description: Binary data

postgres_fdw: incomplete subabort cleanup of connections used in async execution

2021-12-19 Thread Etsuro Fujita
post-subabort case as well. Also, I modified the initialization so that it’s done only if necessary, to save cycles, and improved a comment on the initialization a bit. Attached is a patch for that. Best regards, Etsuro Fujita fix-postgres-fdw-subabort-cleanup.patch Description: Binary data

Re: postgres_fdw: incomplete subabort cleanup of connections used in async execution

2021-12-21 Thread Etsuro Fujita
Hi Alexander, On Wed, Dec 22, 2021 at 1:08 AM Alexander Pyhalov wrote: > Etsuro Fujita писал 2021-12-19 13:25: > > While working on [1], I noticed $SUBJECT: postgres_fdw resets the > > per-connection states of connections, which store async requests sent > > to remote ser

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-01-06 Thread Etsuro Fujita
On Fri, Dec 3, 2021 at 6:07 PM Fujii Masao wrote: > On 2021/11/16 18:55, Etsuro Fujita wrote: > > I changed my mind; I’ll update the patch to ignore the error as > > before, because 1) as far as I know, there are no reports from the > > field concerning that we ignore a

Re: Asynchronous Append on postgres_fdw nodes.

2021-03-19 Thread Etsuro Fujita
On Fri, Mar 19, 2021 at 9:57 PM Justin Pryzby wrote: > is_async_capable_path() should probably have a "break" for case T_ForeignPath. Good catch! Will fix. > little typos: > aready > sigle > givne > a event Lots of typos. :-( Will fix. Thank you for the review! Best regards, Etsuro Fujita

Obsolete comment in postgres_fdw.c

2021-03-29 Thread Etsuro Fujita
While working on async execution, I noticed $subject. Attached is a small patch for that. Best regards, Etsuro Fujita fix-obsolete-comment.patch Description: Binary data

Re: Obsolete comment in postgres_fdw.c

2021-03-29 Thread Etsuro Fujita
On Mon, Mar 29, 2021 at 7:15 PM Etsuro Fujita wrote: > While working on async execution, I noticed $subject. Attached is a > small patch for that. Applied to all supported branches. Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2021-03-30 Thread Etsuro Fujita
his behavior is aggressive. I think it would be better to add more comments, though. Anyway, these are all about naming and docs/comments, so I'll return to this after committing the patch. Thanks for the review! Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2021-03-31 Thread Etsuro Fujita
On Tue, Mar 30, 2021 at 8:40 PM Etsuro Fujita wrote: > I'm happy with the patch, so I'll commit it if there are no objections. Pushed. Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-01 Thread Etsuro Fujita
ER_CACHE_ALWAYS animals will probably be unhappy too, but > they are slower than valgrind. > > (Note that the test case appears to succeed, you have to notice that > the backend crashed after exiting.) Will look into this. Best regards, Etsuro Fujita

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-05 Thread Etsuro Fujita
ExecInitModifyTable > (nodeModifyTable.c:2575) The reason for this would be that epoll_wait() is called with maxevents exceeding the size of the input event array in the test case. To fix, I adjusted the parameters to call the caller function WaitEventSetWait() with in ExecAppendAsyncEventWait(). Patch attached. Best regards, Etsuro Fujita fix-ExecAppendAsyncEventWait-2021-04-05.patch Description: Binary data

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-06 Thread Etsuro Fujita
On Tue, Apr 6, 2021 at 12:01 PM Kyotaro Horiguchi wrote: > At Mon, 5 Apr 2021 17:15:47 +0900, Etsuro Fujita > wrote in > > On Fri, Apr 2, 2021 at 12:09 AM Tom Lane wrote: > > > The buildfarm points out that this fails under valgrind. > > > I easily reproduced i

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-06 Thread Etsuro Fujita
On Tue, Apr 6, 2021 at 5:45 PM Etsuro Fujita wrote: > Also, we should improve the code to avoid > the consistency mentioned above, Sorry, s/consistency/inconsistency/. > I'll apply the patch. Done. Let's see if this works. Best regards, Etsuro Fujita

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-04-08 Thread Etsuro Fujita
On Fri, Apr 9, 2021 at 9:49 AM Justin Pryzby wrote: > > I rebased this patch to resolve a trivial 1 line conflict from c5b7ba4e6. > > -- > Justin

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-04-08 Thread Etsuro Fujita
ontinue to work on it for PG15. Sorry for the empty email. Best regards, Etsuro Fujita

Re: problem with RETURNING and update row movement

2020-07-30 Thread Etsuro Fujita
rt(), in which case the passed-in slot would not store the inserted tuple anymore. To fix these, I modified the patch so that we 1) add to ExecInsert() an output parameter slot to store the inserted tuple, and 2) compute RETURNING based on the parameter. I also added a regression test case. Attached is an updated version of the patch. Sorry for the long delay. Best regards, Etsuro Fujita v3-0001-Fix-a-bug-with-RETURNING-when-UPDATE-moves-tuple.patch Description: Binary data

Re: problem with RETURNING and update row movement

2020-08-02 Thread Etsuro Fujita
the destination partition before calling ExecProcessReturning() if needed, and 2) in ExecProcessReturning(), we only initialize ecxt_scantuple's tts_tableOid when needed, which would save cycles a bit for non-foreign-table-direct-modification cases. Attached is a new version of the patch. Best regards, Etsuro F

Re: problem with RETURNING and update row movement

2020-08-04 Thread Etsuro Fujita
which I've proposed a workaround (attached). > Another is that we forgot in our patch to "materialize" the virtual > tuple after conversion, which means slot_getsysattr() can't find it to > access system columns like xmin, etc. Ok, I’ll look at those closely. Best regards, Etsuro Fujita

Yet another issue with step generation in partition pruning

2020-08-04 Thread Etsuro Fujita
ng this issue. Best regards, Etsuro Fujita [1] https://www.postgresql.org/message-id/CAPmGK15%3Dc8Q-Ac3ogzZp_d6VsfRYSL2tD8zLwy_WYdrMXQhiCQ%40mail.gmail.com fix-pruning-step-generation.patch Description: Binary data

Re: Yet another issue with step generation in partition pruning

2020-08-05 Thread Etsuro Fujita
gt; On Tue, Aug 4, 2020 at 9:46 PM Etsuro Fujita wrote: > > Attached is a patch for fixing this issue. > > I have looked at the patch and played around with it using the > regression tests you've added recently. I was not able to find any > results that looked surprising. Th

Re: Yet another issue with step generation in partition pruning

2020-08-06 Thread Etsuro Fujita
On Thu, Aug 6, 2020 at 12:20 AM Etsuro Fujita wrote: > Will push the patch tomorrow. Done. (I didn't have time for this, because I was terribly busy with other stuff.) Best regards, Etsuro Fujita

Re: Yet another issue with step generation in partition pruning

2020-08-07 Thread Etsuro Fujita
On Fri, Aug 7, 2020 at 2:55 PM Etsuro Fujita wrote: > On Thu, Aug 6, 2020 at 12:20 AM Etsuro Fujita wrote: > > Will push the patch tomorrow. > > Done. (I didn't have time for this, because I was terribly busy with > other stuff.) I mean I didn't have time for thi

Re: problem with RETURNING and update row movement

2020-08-07 Thread Etsuro Fujita
;tts_tid; > > ..but it might be too ugly :(. Yeah, I think that would be a bit ugly, and actually, is not correct in case of postgres_fdw foreign table, in which case Xmin and Cmin are also set to 0 [2]. I think we should probably first address the tableam issue that you pointed out, but I don't think I'm the right person to do so. Best regards, Etsuro Fujita [2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=da7d44b627ba839de32c9409aca659f60324de76

Re: Asynchronous Append on postgres_fdw nodes.

2020-08-14 Thread Etsuro Fujita
On Fri, Aug 14, 2020 at 10:29 AM Thomas Munro wrote: > On Thu, Jul 2, 2020 at 3:20 PM Etsuro Fujita wrote: > > I'd like to join the party, but IIUC, we don't yet reach a consensus > > on which one is the right way to go. So I think we need to discuss > >

Re: Append with naive multiplexing of FDWs

2020-08-31 Thread Etsuro Fujita
> back just a few tuples from each shard, but bad for anything wanting > > to ship back millions of tuples to chew on locally. Do you think > > that'd be useful enough on its own? > > Yes, I think so. There are many data warehouse queries that want to > return only aggregate values, or filter for a small number of rows. > Even OLTP queries might return only a few rows from multiple partitions. > This would allow for a proof-of-concept implementation so we can see how > realistic this approach is. +1 Best regards, Etsuro Fujita

Re: Append with naive multiplexing of FDWs

2020-08-31 Thread Etsuro Fujita
ASC NULLS LAST -> Foreign Scan on public.ft33 pt3_3 (cost=100.00..155.68 rows=1365 width=36) Output: pt3_3.a, pt3_3.b Remote SQL: SELECT a, b FROM public.t33 ORDER BY a ASC NULLS LAST (10 rows) For this query, we can’t apply asynchrony optimization. To disable it for such

Re: Append with naive multiplexing of FDWs

2020-08-31 Thread Etsuro Fujita
On Mon, Aug 31, 2020 at 6:20 PM Etsuro Fujita wrote: > * I know your patch is a POC one, but one concern about it (and > Horiguchi-san's patch set) is concurrent data fetches by multiple > foreign scan nodes using the same connection in the case of > postgres_fdw. Here is an e

Re: Append with naive multiplexing of FDWs

2020-09-06 Thread Etsuro Fujita
On Tue, Sep 1, 2020 at 9:45 AM Kyotaro Horiguchi wrote: > At Mon, 31 Aug 2020 19:10:39 +0900, Etsuro Fujita > wrote in > > On Mon, Aug 31, 2020 at 6:20 PM Etsuro Fujita > > wrote: > > > * I know your patch is a POC one, but one concern about it (and > >

Inconsistent Japanese name order in v13 contributors list

2020-09-08 Thread Etsuro Fujita
Hi, Attached is a patch to standardize Japanese names as given-name-first in the v13 contributors list as before. Best regards, Etsuro Fujita Standardize-Japanese-names.patch Description: Binary data

Re: Inconsistent Japanese name order in v13 contributors list

2020-09-08 Thread Etsuro Fujita
On Wed, Sep 9, 2020 at 2:58 PM tsunakawa.ta...@fujitsu.com wrote: > From: Fujii Masao > > On 2020/09/09 14:15, Etsuro Fujita wrote: > > > Attached is a patch to standardize Japanese names as given-name-first > > > in the v13 contributors list as before. > > &g

Minor cleanup of partbounds.c

2020-09-09 Thread Etsuro Fujita
Here is a patch for minor cleanup of the partbounds.c changes made by commit c8434d64c: 1) removes a useless assignment (in normal builds) and 2) improves comments a little. Best regards, Etsuro Fujita partbounds-cleanup.patch Description: Binary data

Re: Inconsistent Japanese name order in v13 contributors list

2020-09-10 Thread Etsuro Fujita
On Wed, Sep 9, 2020 at 9:16 PM Peter Eisentraut wrote: > On 2020-09-09 07:15, Etsuro Fujita wrote: > > Attached is a patch to standardize Japanese names as given-name-first > > in the v13 contributors list as before. > > Given existing practice, this patch looks okay. I

Re: Minor cleanup of partbounds.c

2020-09-10 Thread Etsuro Fujita
On Thu, Sep 10, 2020 at 2:05 AM Alvaro Herrera wrote: > On 2020-Sep-09, Etsuro Fujita wrote: > > Here is a patch for minor cleanup of the partbounds.c changes made by > > commit c8434d64c: 1) removes a useless assignment (in normal builds) > > LGTM. > > > and 2) impr

postgres_fdw: misplaced? comments in connection.c

2021-10-11 Thread Etsuro Fujita
ansactions * were used. Try to use WARNING where we can. But we don’t use WARNING anywhere in that function. The right place for this is pgfdw_exec_cleanup_query()? Best regards, Etsuro Fujita

Re: postgres_fdw: misplaced? comments in connection.c

2021-10-11 Thread Etsuro Fujita
On Mon, Oct 11, 2021 at 5:05 PM Etsuro Fujita wrote: > The comments for pgfdw_get_cleanup_result() say this: > > * It's not a huge problem if we throw an ERROR here, but if we get into error > * recursion trouble, we'll end up slamming the connection shut, which will &g

Re: postgres_fdw: misplaced? comments in connection.c

2021-10-13 Thread Etsuro Fujita
On Tue, Oct 12, 2021 at 1:33 PM Etsuro Fujita wrote: > On Mon, Oct 11, 2021 at 5:05 PM Etsuro Fujita wrote: > > The comments for pgfdw_get_cleanup_result() say this: > > > > * It's not a huge problem if we throw an ERROR here, but if we get into > > error > &

postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-10-31 Thread Etsuro Fujita
, I think this is useful, so I’ll add this to the upcoming commitfest. Best regards, Etsuro Fujita [1] https://www.postgresql.org/message-id/CAPmGK177E6HPcCQB4-s%2Bm9AcCZDHCC2drZy%2BFKnnvEXw9kXoXQ%40mail.gmail.com postgres-fdw-parallel-commit-1.patch Description: Binary data

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-11-07 Thread Etsuro Fujita
On Mon, Nov 1, 2021 at 3:22 PM Fujii Masao wrote: > On 2021/10/31 18:05, Etsuro Fujita wrote: > > The patch is pretty simple: if a server option added > > by the patch “parallel_commit” is enabled, > > Could you tell me why the parameter is necessary? > Can't we

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-11-07 Thread Etsuro Fujita
RELEASE 0.147 ms, COMMIT 1.305 ms Good to know! Thanks for testing! Best regards, Etsuro Fujita

Re: Commitfest 2021-11 Patch Triage - Part 1

2021-11-07 Thread Etsuro Fujita
ontroversial to add COPY to the FDW API > SQL > spec wise. Amit, Justin and Takayuki-san has done a lot of review and the > patch is marked Ready for Committer. Any committer interested in taking this > on? Tomas? I’d like to work on this unless Tomas (or anyone else) want. Best regards, Etsuro Fujita

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-11-16 Thread Etsuro Fujita
On Mon, Nov 8, 2021 at 1:13 PM Fujii Masao wrote: > On 2021/11/07 18:06, Etsuro Fujita wrote: > > On Mon, Nov 1, 2021 at 3:22 PM Fujii Masao > > wrote: > >> Could you tell me why the parameter is necessary? > >> Can't we always enable the feature? >

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-11-16 Thread Etsuro Fujita
processed the rest of the entries while waiting for the command result, we cannot reduce the total time very much. Maybe I'm missing something, though. Best regards, Etsuro Fujita

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2021-11-18 Thread Etsuro Fujita
On Thu, Nov 18, 2021 at 1:09 PM Fujii Masao wrote: > On 2021/11/16 18:55, Etsuro Fujita wrote: > > Sorry, my explanation was not enough, but I don’t think this is always > > true. Let me explain using an example: > > > > create server loopback foreign data w

Re: ON CONFLICT DO UPDATE for partitioned tables

2018-03-23 Thread Etsuro Fujita
(2018/03/22 18:31), Amit Langote wrote: On 2018/03/20 20:53, Etsuro Fujita wrote: Here are comments on executor changes in (the latest version of) the patch: @@ -421,8 +424,18 @@ ExecInsert(ModifyTableState *mtstate, ItemPointerData conflictTid; bool

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-03-23 Thread Etsuro Fujita
(2018/03/23 4:09), Robert Haas wrote: On Tue, Feb 27, 2018 at 7:01 AM, Etsuro Fujita Attached is a new version of the patch set. I took a look at this patch set today but I really don't think we should commit something so minimal. It's got at least four issues that I see: 1

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-03-23 Thread Etsuro Fujita
(2018/03/23 21:02), Robert Haas wrote: On Fri, Mar 23, 2018 at 7:55 AM, Etsuro Fujita wrote: Maybe I'm missing something, but I think the proposed FDW API could be used for the COPY case as well with some modifications to core. If so, my question is: should we support COPY into fo

Oddity in COPY FROM handling of check constraints on partition tables

2018-03-27 Thread Etsuro Fujita
ts, not the partition's constraints, when checking the constraint against the input row. Attached is a patch for fixing this issue. Best regards, Etsuro Fujita *** a/src/backend/commands/copy.c --- b/src/backend/commands/copy.c *** *** 2709,2715 CopyFrom(CopyState

Re: Oddity in COPY FROM handling of check constraints on partition tables

2018-03-28 Thread Etsuro Fujita
(2018/03/28 10:28), Amit Langote wrote: >> Attached is a patch for fixing this issue. > > That looks good to me. This one would need to be back-patched to v10. Thanks for the review! Best regards, Etsuro Fujita

Re: Oddity in COPY FROM handling of check constraints on partition tables

2018-03-28 Thread Etsuro Fujita
e can not add this to v11 open items since it isn't a v11 bug exactly. OK, done. Best regards, Etsuro Fujita

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-03-30 Thread Etsuro Fujita
(2018/03/19 20:25), Amit Langote wrote: On 2018/02/27 21:01, Etsuro Fujita wrote: Attached is a new version of the patch set. * Comments postgres-fdw-refactoring-1.patch: 1. Just a minor nitpick: wouldn't it be better to call it create_foreign_modify_state just like its "finish&qu

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-03-30 Thread Etsuro Fujita
(2018/03/23 20:55), Etsuro Fujita wrote: (2018/03/23 4:09), Robert Haas wrote: 1. It still doesn't work for COPY, because COPY isn't going to have a ModifyTableState. I really think it ought to be possible to come up with an API that can handle both INSERT and COPY; I don't thi

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-03-30 Thread Etsuro Fujita
t now. Thank you for the review! Best regards, Etsuro Fujita

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-02 Thread Etsuro Fujita
(2018/03/30 22:28), Alvaro Herrera wrote: Etsuro Fujita wrote: Now we have ON CONFLICT for partitioned tables, which requires the conversion map to be computed in ExecInitPartitionInfo, so I updated the patch so that we keep the former step in ExecInitPartitionInfo and

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-02 Thread Etsuro Fujita
but it could perhaps be avoided). I agree. Thanks for the review! Best regards, Etsuro Fujita

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-03 Thread Etsuro Fujita
(2018/04/03 13:32), Amit Langote wrote: On 2018/04/02 21:26, Etsuro Fujita wrote: We wouldn't need that for foreign partitions (When DO NOTHING with an inference specification or DO UPDATE on a partitioned table containing foreign partitions, the planner would throw an error before we g

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-03 Thread Etsuro Fujita
(2018/04/03 13:59), Amit Langote wrote: On 2018/04/02 21:29, Etsuro Fujita wrote: (2018/04/02 18:49), Amit Langote wrote: I looked at the new patch. It looks good overall, although I have one question -- IIUC, BeginForeignInsert() performs actions that are equivalent of performing

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-04 Thread Etsuro Fujita
(2018/04/03 22:01), Etsuro Fujita wrote: Attached is an updated version of the patch. Patch foreign-routing-fdwapi-3.patch is created on top of patch postgres-fdw-refactoring-3.patch and the bug-fix patch [1]. One thing I noticed about patch foreign-routing-fdwapi-3.patch is this bug: the

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-04 Thread Etsuro Fujita
(2018/04/04 19:31), Etsuro Fujita wrote: Attached is an updated version of the patch set: * As before, patch foreign-routing-fdwapi-4.patch is created on top of patch postgres-fdw-refactoring-4.patch and the bug-fix patch [1]. I did a bit of cleanup and comment-rewording to patch foreign

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-04 Thread Etsuro Fujita
(2018/04/05 15:37), Amit Langote wrote: On 2018/04/05 15:02, Etsuro Fujita wrote: (2018/04/04 19:31), Etsuro Fujita wrote: Attached is an updated version of the patch set: * As before, patch foreign-routing-fdwapi-4.patch is created on top of patch postgres-fdw-refactoring-4.patch and the bug

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-05 Thread Etsuro Fujita
patch and 0002_copy-from-check-constraint-fix.patch. Best regards, Etsuro Fujita [1] https://www.postgresql.org/message-id/5aba4074.1090...@lab.ntt.co.jp *** a/contrib/postgres_fdw/postgres_fdw.c --- b/contrib/postgres_fdw/postgres_fdw.c *** *** 376,387 static bool ec_member_matche

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

2018-04-09 Thread Etsuro Fujita
explain in full aggressiveness (i.e. including costs etc) and enables debug3 logging could help narrow this down? +1 because we cannot deny the possibility that the plan instability is caused by such an unexpected behavior of postgres_fdw. Best regards, Etsuro Fujita

Re: [HACKERS] Add support for tuple routing to foreign partitions

2018-04-09 Thread Etsuro Fujita
(2018/04/07 8:25), Robert Haas wrote: On Thu, Apr 5, 2018 at 6:21 AM, Etsuro Fujita wrote: Attached is an updated version of the patch set plus the patch in [1]. Patch 0003_foreign-routing-fdwapi-6.patch can be applied on top of patch 0001_postgres-fdw-refactoring-6.patch and 0002_copy-from

Re: Instability in the postgres_fdw regression test

2018-04-12 Thread Etsuro Fujita
the normal range of "C 1" values, so that whether it's still visible to get_actual_variable_range has less effect on this selectivity estimate. That's a hack, for sure, but I don't see any other fix that's much less of a hack. Thanks for the detective work. Thanks a lot! Best regards, Etsuro Fujita

Oddity in tuple routing for foreign partitions

2018-04-16 Thread Etsuro Fujita
ddressing this, I'll post an updated version of the fix-postgres_fdw-WCO-handling patch.) Best regards, Etsuro Fujita *** a/contrib/postgres_fdw/expected/postgres_fdw.out --- b/contrib/postgres_fdw/expected/postgres_fdw.out *** *** 7454,7459 select tableoid::regclass, * FRO

Re: Oddity in tuple routing for foreign partitions

2018-04-16 Thread Etsuro Fujita
Hi Amit, (2018/04/17 10:10), Amit Langote wrote: > On 2018/04/16 20:25, Etsuro Fujita wrote: >> While updating the fix-postgres_fdw-WCO-handling patch, I noticed that >> the patch for tuple routing for foreign partitions doesn't work well >> with remote tri

  1   2   3   4   5   6   7   8   9   >