BEAM-2217 NotImplementedError - DataflowRunner parsing Protos from PubSub (Python SDK)

2020-06-08 Thread Lien Michiels
Hi everyone, First time writing the email list, so please tell me if I'm doing this all wrong. I'm building a streaming pipeline to be run on the DataflowRunner that reads from PubSub and writes to BQ using the Python 3 SDK. I can get the pipeline started fine with the DirectRunner, but as soon

Re: BEAM-2217 NotImplementedError - DataflowRunner parsing Protos from PubSub (Python SDK)

2020-06-09 Thread Lien Michiels
es to the way we use typehints in the most recent > Beam release (2.21) that might be causing this [1]. If you're using 2.21 > could you try reverting to the old behavior (call > `apache_beam.typehints.disable_type_annotations()` before constructing the > pipeline) to see if that helps?

Re: BEAM-2217 NotImplementedError - DataflowRunner parsing Protos from PubSub (Python SDK)

2020-06-11 Thread Lien Michiels
on for to_type_hint >> - DataflowRunner calls from_runner_api, which I believe is a workaround >> we can eventually remove (+Robert Bradshaw ), which >> in turn tries to get type hints for every coder in the pipeline >> >> Brian >> >> On Tue, Jun 9, 2020 at 1: