Re: Integration Alluxio and Flink

Hi Robert, I've missed to tell that I built a fat-jar of the job using `sbt assembly` so my job is including alluxio-core-client. 2016-03-15 17:45 GMT+01:00 Robert Metzger : > Hi Andrea, > > the filesystem class can not be in the job jar. You have to put it into > the lib folder. > > On Tue, Ma

Re: Integration Alluxio and Flink

Hi Andrea, the filesystem class can not be in the job jar. You have to put it into the lib folder. On Tue, Mar 15, 2016 at 5:40 PM, Andrea Sella wrote: > Hi Till, > > I put the jar as dependency of my job on build.sbt. I need to do > somenthing else? > > val flinkDependencies = Seq( > "org.ap

Re: Integration Alluxio and Flink

Hi Till, I put the jar as dependency of my job on build.sbt. I need to do somenthing else? val flinkDependencies = Seq( "org.apache.flink" %% "flink-scala" % flinkVersion % "provided", "org.apache.flink" %% "flink-streaming-scala" % flinkVersion % "provided", ("org.alluxio" % "alluxio-core-

Re: Integration Alluxio and Flink

Hi Andrea, can it be that the alluxio.hadoop.FileSystem is not in your classpath? Have you put the respective jar file in Flink’s lib folder? Cheers, Till ​ On Tue, Mar 15, 2016 at 12:55 PM, Andrea Sella wrote: > Hi Till, > > I've tried your suggestion (source-code >

Re: Integration Alluxio and Flink

Hi Till, I've tried your suggestion (source-code ) and now it throws: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class alluxio.hadoop.FileSystem not found. The core-site.xml has been set correctly and into the alluxio-wordcount jar

Re: Integration Alluxio and Flink

Hi Andrea, the problem won’t be netty-all but netty, I suspect. Flink is using version 3.8 whereas alluxio-core-client uses version 3.2.2. I think you have to exclude or shade this dependency away. Cheers, Till ​ On Mon, Mar 14, 2016 at 5:12 PM, Andrea Sella wrote: > Hi Till, > I tried to down

Re: Integration Alluxio and Flink

Hi Till, I tried to downgrade the Alluxio's netty version from 4.0.28.Final to 4.0.27.Final to align Flink and Alluxio dependencies. First of all, Flink 1.0.0 uses 4.0.27.Final, is it correct? Btw it doesn't work, same error as above. BR, Andrea 2016-03-14 15:30 GMT+01:00 Till Rohrmann : > Yes i

Re: Integration Alluxio and Flink

Yes it seems as if you have a netty version conflict. Maybe the alluxio-core-client.jar pulls in an incompatible netty version. Could you check whether this is the case? But maybe you also have another dependencies which pulls in a wrong netty version, since the Alluxio documentation indicates that

Integration Alluxio and Flink

Hi to all, I'm trying to integrate Alluxio and Apache Flink, I followed Running Flink on Alluxio to setup Flink. I tested in local mode executing: bin/flink run ./examples/batch/WordCount.jar --input alluxio:///flink/README.