> * Kouhei Kaigai (kai...@ak.jp.nec.com) wrote: > > I (plan to) use custom-scan of course. Once a relation is referenced > > and optimizer decided GPU acceleration is cheaper, associated custom- > > scan node read the data from underlying relation (or in-memory cache > > if exists) then move to the shared memory buffer to deliver GPU > > management background worker that launches asynchronous DMA one by one. > > After that, custom-scan node receives filtered records via shared- > > memory buffer, so it can construct tuples to be returned to the upper > > node. > > Alright- but have you discussed this with Robert? We're going to be > whacking things around for parallel support with new nodes and more built-in > helper functionality for doing this work and I'm not anxious to have > CustomScan end up being a legacy interface that we're required to pull > forward because we accepted it before things had settled. > I had briefly introduced him my idea using GPU at Ottawa last year, even though I'm not certain he remembered it. At least, idea of custom-scan node came from the discussion at that time.
> > The custom-scan API is thin abstraction towards the plan node > > interface, not tightly convinced with a particular use case, like GPU, > > remote-join and so on. So, I'm quite optimistic for the future > maintainability. > > I don't see how you can be when there hasn't been any discussion that I've > seen about how parallel query execution is going to change things for us. > If parallel query execution changes whole of the structure of plan nodes, it will also affect to the interface of custom-scan because it is a thin- abstraction of plan-node. However, if parallel execution feature is implemented as one of new plan node in addition to existing one, I cannot imagine a scenario that affects to the structure of another plan node. > > Also, please remind the discussion at the last developer meeting. > > The purpose of custom-scan (we didn't name it at that time) is to > > avoid unnecessary project branch for people who want to implement > > their own special feature but no facilities to enhance > > optimizer/executor are supported. > > Even though we have in-core parallel execution feature by CPU, it also > > makes sense to provide some unique implementation that may be suitable > > for a specific region. > > The issue here is that we're going to be expected to maintain an interface > once we provide it and so that isn't something we should be doing lightly. > Particularly when it's as involved as this kind of change is with what's > going on in the backend where we are nearly 100% sure to be changing things > in the next release or two. > FDW APIs are also revised several times in the recent releases. If we can design "perfect" interface from the beginning, it's best but usually hard to design. Also, custom-scan interface is almost symmetric with existing plan node structures, so its stability is relatively high, I think. Thanks, -- NEC OSS Promotion Center / PG-Strom Project KaiGai Kohei <kai...@ak.jp.nec.com> -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
