Re: NoClassDefFoundError with ElasticsearchSink on Yarn

2016-09-01 Thread aris kol
Classic problem of every uber-jar containing Hadoop dependencies and being deployed on Yarn. What actually happens is that some Hadoop dependency relies on an old version of guava (11 in this case), which doesn't have the method. You may have assembled your fat-jar properly, but because Hadoop

Re: Accessing state in connected streams

2016-08-31 Thread aris kol
you need more details. Cheers, Aljoscha On Tue, 30 Aug 2016 at 16:21 aris kol mailto:gizera...@hotmail.com>> wrote: Hi Aljoscha, I removed business objects and logic etc.. I am happy to post here [??] I am sure this is a common issue when you start to seriously mess with state.

Re: Accessing state in connected streams

2016-08-30 Thread aris kol
using a CoFlatMap for this. Could you maybe post the code of your CoFlatMapFunction (or you could send it to me privately if you have concerns with publicly posting it) then I could have a look. Cheers, Aljoscha On Mon, 29 Aug 2016 at 15:48 aris kol mailto:gizera...@hotmail.com>> wrote: An

Re: Accessing state in connected streams

2016-08-29 Thread aris kol
Any other opinion on this? Thanks :) Aris From: aris kol Sent: Sunday, August 28, 2016 12:04 AM To: user@flink.apache.org Subject: Re: Accessing state in connected streams In the implementation I am passing just one CoFlatMapFunction, where flatMap1, which

Re: Accessing state in connected streams

2016-08-27 Thread aris kol
/api/datastream/ConnectedStreams.html Sameer On Sat, Aug 27, 2016 at 6:13 PM, aris kol mailto:gizera...@hotmail.com>> wrote: Let's say I have two types sharing the same trait trait Event { def id: Id } case class EventA(id: Id, info: InfoA) extends Event case class EventB(id: Id, info: InfoB) ex

Re: Accessing state in connected streams

2016-08-27 Thread aris kol
are unprocessed elements from stream B when elements arrive from stream A. You might need to do that for elements from both streams depending on how you use them. You will get NPE if you assume events have arrived from A and but they might be lagging behind. On Sat, Aug 27, 2016 at 6:13 PM, aris kol

Accessing state in connected streams

2016-08-27 Thread aris kol
Let's say I have two types sharing the same trait trait Event { def id: Id } case class EventA(id: Id, info: InfoA) extends Event case class EventB(id: Id, info: InfoB) extends Event Each of these events gets pushed to a Kafka topic and gets consumed by a stream in Flink. Let's say I have two

RE: 1.1 release

2016-07-18 Thread aris kol
tial release discussion: > https://mail-archives.apache.org/mod_mbox/flink-dev/201607.mbox/%3c577637ed.2090...@apache.org%3e > > > On Mon, Jul 18, 2016 at 3:30 PM, aris kol wrote: > > Hi, > > > > Any clues as to when 1.1 will be released? > > > > Thank, > > Aris > >

1.1 release

2016-07-18 Thread aris kol
Hi, Any clues as to when 1.1 will be released? Thank,Aris

1.1-snapshot issues

2016-05-17 Thread aris kol
Hi guys, Since yesterday, I am getting this: [warn] apache.snapshots: tried[warn] http://repository.apache.org/snapshots/org/apache/flink/flink-scala_2.11/1.1-SNAPSHOT/flink-scala_2.11-1.1-SNAPSHOT.pom[error] SERVER ERROR: Proxy Error url=http://repository.apache.org/snapshots/org/apache/

RE: classpath issue on yarn

2016-04-28 Thread aris kol
. On Tue, Apr 26, 2016 at 12:28 PM, aris kol wrote: Hi guys,I ran into a weird classpath issue while running a streaming job on a yarn cluster.I have a relatively simple flow that reads data from kafka, does a few manipulations and then indexes them on Elasticsearch (2.3).I use the

RE: classpath issue on yarn

2016-04-26 Thread aris kol
/flink-1.0.2/flink-1.0.2-bin-hadoop2-scala_2.11.tgz has a clean flink-dist jar. On Tue, Apr 26, 2016 at 12:28 PM, aris kol wrote: Hi guys,I ran into a weird classpath issue while running a streaming job on a yarn cluster.I have a relatively simple flow that reads data from kafka, does a few

classpath issue on yarn

2016-04-26 Thread aris kol
Hi guys,I ran into a weird classpath issue while running a streaming job on a yarn cluster.I have a relatively simple flow that reads data from kafka, does a few manipulations and then indexes them on Elasticsearch (2.3).I use the elasticsearch2 connector (1.1-SNAPSHOT) (bleeding edge, I know).T