Re: join two streams with pyflink

2024-04-02 Thread Biao Geng
Hi Thierry, Your case is not very complex and I believe all programming language(e.g. Java, Python, SQL) interfaces of flink can do that. When using pyflink, you can use pyflink datastream/table/SQL API. Here are some examples of using pyflink table api: https://nightlies.apache.org/flink/flink-do

join two streams with pyflink

2024-04-02 Thread Fokou Toukam, Thierry
Hi, i have 2 streams as sean in this example (6> {"tripId": "275118740", "timestamp": "2024-04-02T06:20:00Z", "stopSequence": 13, "stopId": "61261", "bearing": 0.0, "speed": 0.0, "vehicleId": "39006", "routeId": "747"} 1> {"visibility": 1, "weather_conditions": "clear sky", "timestamp": "20

Re: Join two streams

2023-06-30 Thread Иван Борисов
windowing by event time, all the sources need to emit watermarks. > - watermarks are the logical clock used when event-time timing > - you could use either processing time windows, or adjust watermark strategy > of your sources accordingly > > ... didn't check other potential s

RE: Join two streams

2023-06-29 Thread Schwalbe Matthias
accordingly ... didn't check other potential sources of troubles in your code Hope this helps Thias -Original Message- From: Иван Борисов Sent: Freitag, 30. Juni 2023 05:45 To: user@flink.apache.org Subject: Join two streams Hello, plz help me, I can't join two streams. In the joi

Re: Join two streams

2023-06-29 Thread Shrihari R
9:16 AM Иван Борисов wrote: > Hello, > plz help me, I can't join two streams. In the joined stream I've got > zero messages and can't understand why? > > Kafka Topics: > 1st stream > topic1: {'data': {'temp':25.2, 'sensore_name': &

Join two streams

2023-06-29 Thread Иван Борисов
Hello, plz help me, I can't join two streams. In the joined stream I've got zero messages and can't understand why? Kafka Topics: 1st stream topic1: {'data': {'temp':25.2, 'sensore_name': 'T1', 'timestamp': 123123131}, 'comp

Re: Join two streams from Kafka

2021-02-11 Thread Arvid Heise
Hi Shamit, unless you have some temporal relationship between the records to be joined, you have to use a regular join over stream 1 and stream 2. Since you cannot define any window, all data will be held in Flink's state, which is not an issue for a few millions but probably means you have to use

Join two streams from Kafka

2021-02-09 Thread Shamit
Hello Flink Users, I am newbie and have question on join of two streams (stream1 and stream2 ) from Kafka topic based on some key. In my use case I need to join with stream2 data which might be year old and more. Now if on stream1 the data gets arrived today and I need to join with stream2 base

Re: Join two streams using a count-based window

2016-06-10 Thread Nikos R. Katsipoulakis
You might be interested in > > https://cwiki.apache.org/confluence/display/FLINK/Data+exchange+between+tasks > > -Matthias > > On 06/09/2016 08:13 PM, Nikos R. Katsipoulakis wrote: > > Hello all, > > > > At first, I have a question posted on > > > http://stac

Re: Join two streams using a count-based window

2016-06-10 Thread Matthias J. Sax
: > Hello all, > > At first, I have a question posted on > http://stackoverflow.com/questions/37732978/join-two-streams-using-a-count-based-window > . I am re-posting this on the mailing list in case some of you are not > on SO. > > In addition, I would like to know what is the

Join two streams using a count-based window

2016-06-09 Thread Nikos R. Katsipoulakis
Hello all, At first, I have a question posted on http://stackoverflow.com/questions/37732978/join-two-streams-using-a-count-based-window . I am re-posting this on the mailing list in case some of you are not on SO. In addition, I would like to know what is the difference between Flink and other