As of Github PR #1998 (https://github.com/apache/beam/pull/1998), the new
Pipeline Surgery API is ready and available. There are a couple of
refinements coming in PR #2006, but in general pipelines can now, post
construction, have PTransforms swapped out to whatever the runner desires
(standard "behavior-maintaining" caveats apply).

Moving forwards, this will enable pipelines to be run on multiple runners
without having to reconstruct the graph via repeated applications of
PTransforms to the pipeline (this also includes being able to, for example,
read a pipeline from a serialized representation, and executing the result
on an arbitrary runner).

Those of you who are runner authors (at least, those who I can easily
identify as such) should expect a Pull Request from me sometime next week
porting you off of intercepting calls to apply and to the new surgery API.
You are, of course, welcome to beat me to the punch.

Thanks,

Thomas

Reply via email to