Re: Python pipeline: unable to read from Kafka in a Flink cluster with the expansion service

2023-01-06 Thread Lydian
Hi, I had the same issue before, and I *think* the 8097 is only work if you're using the flink runner, i.e., --runner=FlinkRunner, but it's not working for PortableRunner. So far, what seems work for me is to do things like this: from apache_beam.io.kafka import default_io_expansion_service ReadF

Re: Python pipeline: unable to read from Kafka in a Flink cluster with the expansion service

2023-01-06 Thread Florentin Hennecker
Hi Lydian, and thanks for answering. If you have a working minimal example, could you share it? I’ve tried your suggestion, which required me to bump apache-beam to 2.41. I get a different error when I do that (I tried extracting the relevant stack trace bits): 2023-01-06 10:38:04,885 WARN or