Streaming left outer join

Hello, I'm trying to implement a left outer join of two Kafka streams within a sliding window. So far I have the following code: foos .coGroup(bars) .where(_.baz).equalTo(_.baz) .window(SlidingTimeWindows.of(Time.of(1, TimeUnit.MINUTES), Time.of(1, TimeUnit.SECONDS))) .apply((fs: Iterator

Re: Streaming left outer join

me > know if you need some pointers about that one. > > Cheers, > Aljoscha > > On 26 Jan 2016, at 19:32, Alexander Gryzlov > wrote: > > > > Hello, > > > > I'm trying to implement a left outer join of two Kafka streams within a > sliding window

Re: Streaming left outer join

.com/apache/flink/pull/1527 > > Stephan > > > On Wed, Jan 27, 2016 at 2:06 PM, Alexander Gryzlov > wrote: > >> Hello Aljoscha, >> >> Indeed, it seems like I'd need a custom operator. I imagine this >> involves >> implementing >> org.apache.

Flink + Kafka + Scalabuff issue

Hello, Has anyone tried using ScalaBuff (https://github.com/SandroGrzicic/ScalaBuff) with Flink? We’re trying to consume Protobuf messages from Kafka 0.8 and have hit a performance issue. We run this code: https://gist.github.com/clayrat/05ac17523fcaa52fcc5165d9edb406b8 (where Foo is pre-gene

Re: Flink + Kafka + Scalabuff issue

ch Flink version are you using for this? >> >> (2) Can you also get a heap dump after the job slows down? Slow downs >> like this are often caused by some component leaking memory, maybe in >> Flink, maybe the Scalabuff deserializer. Can you also share the Foo >&g

synchronizing two streams

Hello, We're implementing a streaming outer join operator based on a TwoInputStreamOperator with an internal buffer. In our use-case only the items whose timestamps are within a several-second interval of each other can join, so we need to synchronize the two input streams to ensure maximal yield.

Re: synchronizing two streams

ks for each stream. > > So you can sync both stream within your outer join operator you plan to > implement. > > -Matthias > > On 05/11/2016 05:00 PM, Alexander Gryzlov wrote: > > Hello, > > > > We're implementing a streaming outer join operator based on a &g

Re: synchronizing two streams

or all records that got not joined, before removing > them emit a "record-null" (or "null-record") result tuple. > > No need to block/sleep. > > Does this make sense? > > > -Matthias > > > On 05/12/2016 02:51 PM, Alexander Gryzlov wrote: > > Hmm