Re: Adding a StepMetadataRegistry for Python SDK

2018-03-30 Thread Lukasz Cwik
+1 on minimizing creating new stuff that will be deleted but if it gets us to that goal faster it can still be worthwhile. On Thu, Mar 29, 2018 at 5:51 PM Robert Bradshaw wrote: > If I understand correctly, this is something runner-specific that would > live solely on the runner side (i.e. over

Re: Adding a StepMetadataRegistry for Python SDK

2018-03-29 Thread Robert Bradshaw
If I understand correctly, this is something runner-specific that would live solely on the runner side (i.e. over the Fn API we'd still have a single name for operations rather than pushing this complexity into that protocol as well which I'd really like to avoid, right?) If that's the case, then i

Adding a StepMetadataRegistry for Python SDK

2018-03-28 Thread Pablo Estrada
Hello all, I've filed https://issues.apache.org/jira/browse/BEAM-3955, to consider the possibility of adding some sort of facility to translate different names for the runners. This is currently a problem in Dataflow, where steps can have different names in the backend and in the SDK. This is obser