Re: Beam/Python/Flink unable to deserialize UnboundedSource

2020-06-08 Thread Pradip Thachile
Another data point folks: I've been able to run my simple test case with Dataflow without issue, but Flink is still problematic. Le dim. 7 juin 2020 à 18:48, Pradip Thachile a écrit : > I've attached below some minimal sample code that reproduces this issue > below. This works perfectly with the

Re: Beam/Python/Flink unable to deserialize UnboundedSource

2020-06-07 Thread Pradip Thachile
I've attached below some minimal sample code that reproduces this issue below. This works perfectly with the DirectRunner. -Pradip #!/usr/bin/env python3 import apache_beam as beam import logging import os class DummyPipeline(beam.PTransform): def expand(self, p): ( p

Beam/Python/Flink unable to deserialize UnboundedSource

2020-06-07 Thread Pradip Thachile
Hey folks, I've got a Beam/Python pipeline that works on the DirectRunner and now am trying to run this on a local dev Flink cluster. Running this yields an error out the gate around not being able to deserialize UnboundedSource (my PubSub source). I'm not sure how to debug this and would love to