Re: Accessing state in connected streams

2016-08-31 Thread aris kol
:48 PM To: user@flink.apache.org Subject: Re: Accessing state in connected streams Ah I see, I'm afraid StatefulFunction is more of an internal implementation detail that cannot be used like that. This is a small example that shows how you could do a stateful Co-FlatMap function: o

Re: Accessing state in connected streams

2016-08-30 Thread Aljoscha Krettek
>} > >flatMap(in, out) > > } > } > } > > > applyWithState throws the exception and my intuition says I am doing > seriously wrong in the instantiation. I tried to make something work using > the mapWithState implementation as a guide and I ended up here. > &g

Re: Accessing state in connected streams

2016-08-30 Thread aris kol
mapWithState implementation as a guide and I ended up here. Thanks, Aris From: Aljoscha Krettek Sent: Tuesday, August 30, 2016 10:06 AM To: user@flink.apache.org Subject: Re: Accessing state in connected streams Hi Aris, I think you're on the right track with

Re: Accessing state in connected streams

2016-08-30 Thread Aljoscha Krettek
aris kol wrote: > Any other opinion on this? > > > Thanks :) > > Aris > *From:* aris kol > *Sent:* Sunday, August 28, 2016 12:04 AM > > *To:* user@flink.apache.org > *Subject:* Re: Accessing state in connected streams > > In the implementation I am passing

Re: Accessing state in connected streams

2016-08-29 Thread aris kol
Any other opinion on this? Thanks :) Aris From: aris kol Sent: Sunday, August 28, 2016 12:04 AM To: user@flink.apache.org Subject: Re: Accessing state in connected streams In the implementation I am passing just one CoFlatMapFunction, where flatMap1, which

Re: Accessing state in connected streams

2016-08-27 Thread aris kol
ll want to flatten afterwards before pushing dowstream. Aris From: Sameer W Sent: Saturday, August 27, 2016 11:40 PM To: user@flink.apache.org Subject: Re: Accessing state in connected streams Ok sorry about that :-). I misunderstood as I am not familiar with

Re: Accessing state in connected streams

2016-08-27 Thread Sameer W
Ok sorry about that :-). I misunderstood as I am not familiar with Scala code. Just curious though how are you passing two MapFunction's to the flatMap function on the connected stream. The interface of ConnectedStream requires just one CoMap function- https://ci.apache.org/projects/flink/flink-doc

Re: Accessing state in connected streams

2016-08-27 Thread aris kol
ntB, which is fine and can be fixed - I am doing something completely wrong that Flink doesn't support. Thanks, Aris From: Sameer W Sent: Saturday, August 27, 2016 10:17 PM To: user@flink.apache.org Subject: Re: Accessing state in connected strea

Re: Accessing state in connected streams

2016-08-27 Thread Sameer W
There is no guarantee about the order in which each stream elements arrive in a connected streams. You have to check if the elements have arrived from Stream A before using the information to process elements from Stream B. Otherwise you have to buffer elements from stream B and check if there are

Accessing state in connected streams

2016-08-27 Thread aris kol
Let's say I have two types sharing the same trait trait Event { def id: Id } case class EventA(id: Id, info: InfoA) extends Event case class EventB(id: Id, info: InfoB) extends Event Each of these events gets pushed to a Kafka topic and gets consumed by a stream in Flink. Let's say I have two