[Meetup] Wednesday 17th March 9am PDT / 5pm CET - Emily Ye on Custom Containers

2021-03-15 Thread Max King
Hi Everyone, On Wednesday there will be an online Beam meetup with Emily Ye. Join Emily at 9am PDT / 5pm CET where she will introduce you to Custom Containers, a new feature in the Beam Portability Framework. Register for this free online meetup here: https://pretix.eu/plainschwarz/beammeetup

Re: [Meetup] Wednesday 17th March 9am PDT / 5pm CET - Emily Ye on Custom Containers

2021-03-15 Thread Ahmet Altay
This is great and could we share this on our Twitter account? It will reach more users that way. /cc @Brittany Hermann @Pablo Estrada On Mon, Mar 15, 2021 at 7:32 AM Max King wrote: > Hi Everyone, > > On Wednesday there will be an online Beam meetup with Emily Ye. Join > Emily at 9am PDT / 5p

Re: Apache Beam Python SDK ReadFromKafka does not receive data

2021-03-15 Thread Sumeet Malhotra
Hi Cham, Do you have pointers on what might be going on? Or something else I can try? I had posted the same on StackOverflow [1], it seems that I'm not the only one seeing this issue at the moment. Thanks, Sumeet [1] https://stackoverflow.com/questions/66151919/apache-beam-python-sdk-readfromkaf

Re: Apache Beam Python SDK ReadFromKafka does not receive data

2021-03-15 Thread Chamikara Jayalath
I'm not too familiar with Flink but it seems like, for streaming pipelines, messages from Kafka/SDF read do not get pushed to subsequent steps for some reason. * X-lang Bounded read with Flink seems to be fine. * X-lang Kafka sink and with Flink to be fine. Created https://issues.apache.org/jira/b

Re: Apache Beam Python SDK ReadFromKafka does not receive data

2021-03-15 Thread Sumeet Malhotra
Thanks Cham! But I don't think this is Flink specific. I have observed similar behaviour with DirectRunner as well BTW. ..Sumeet On Tue, Mar 16, 2021 at 12:00 PM Chamikara Jayalath wrote: > I'm not too familiar with Flink but it seems like, for streaming > pipelines, messages from Kafka/SDF rea

Re: Apache Beam Python SDK ReadFromKafka does not receive data

2021-03-15 Thread Chamikara Jayalath
I don't believe Fn API DirectRunner supports streaming yet (I might be wrong). I can confirm that this works for Dataflow. Thanks, Cham On Mon, Mar 15, 2021 at 11:37 PM Sumeet Malhotra wrote: > Thanks Cham! But I don't think this is Flink specific. I have observed > similar behaviour with Direc