On Mon, Feb 1, 2021 at 12:06 PM Etsuro Fujita <etsuro.fuj...@gmail.com> wrote: > On Tue, Nov 17, 2020 at 6:56 PM Etsuro Fujita <etsuro.fuj...@gmail.com> wrote: > > * I haven't yet done anything about the issue on postgres_fdw's > > handling of concurrent data fetches by multiple ForeignScan nodes > > (below *different* Append nodes in the query) using the same > > connection discussed in [2]. I modified the patch to just disable > > applying this feature to problematic test cases in the postgres_fdw > > regression tests, by a new GUC enable_async_append. > > A solution for the issue would be a scheduler designed to handle such > data fetches more efficiently, but I don’t think it’s easy to create > such a scheduler. Rather than doing so, I'd like to propose to allow > FDWs to disable async execution of them in problematic cases by > themselves during executor startup in the first cut. What I have in > mind for that is: > > 1) For an FDW that has async-capable ForeignScan(s), we allow the FDW > to record, for each of the async-capable and non-async-capable > ForeignScan(s), the information on a connection to be used for the > ForeignScan into EState during BeginForeignScan(). > > 2) After doing ExecProcNode() to each SubPlan and the main query tree > in InitPlan(), we give the FDW a chance to a) reconsider, for each of > the async-capable ForeignScan(s), whether the ForeignScan can be > executed asynchronously as planned, based on the information stored > into EState in #1, and then b) disable async execution of the > ForeignScan if not.
s/ExecProcNode()/ExecInitNode()/. Sorry for that. I’ll post an updated patch for this in a few days. Best regards, Etsuro Fujita
