Hi Andrea,
the filesystem class can not be in the job jar. You have to put it into the
lib folder.
On Tue, Mar 15, 2016 at 5:40 PM, Andrea Sella <andrea.se...@radicalbit.io>
wrote:
> Hi Till,
>
> I put the jar as dependency of my job on build.sbt. I need to do
> somenthing else?
>
> val flinkDependencies = Seq(
> "org.apache.flink" %% "flink-scala" % flinkVersion % "provided",
> "org.apache.flink" %% "flink-streaming-scala" % flinkVersion %
> "provided",
> ("org.alluxio" % "alluxio-core-client" % "1.0.0").
> exclude("org.jboss.netty", "netty").
> exclude("io.netty", "netty").
> exclude("io.netty", "netty-all").
> exclude("org.slf4j", "slf4j-api").
> exclude("commons-beanutils", "commons-beanutils-core").
> exclude("commons-collections", "commons-collections").
> exclude("commons-logging", "commons-logging").
> exclude("com.esotericsoftware.minlog", "minlog").
> exclude("org.apache.hadoop","hadoop-yarn-common")
> )
>
> `show compile:dependencyClasspath` shows alluxio-core-client as dependency.
>
> https://github.com/alkagin/alluxio-wordcount/
>
> Thanks,
> Andrea
>
>
>
> 2016-03-15 17:09 GMT+01:00 Till Rohrmann <trohrm...@apache.org>:
>
>> Hi Andrea,
>>
>> can it be that the alluxio.hadoop.FileSystem is not in your classpath?
>> Have you put the respective jar file in Flink’s lib folder?
>>
>> Cheers,
>> Till
>>
>>
>> On Tue, Mar 15, 2016 at 12:55 PM, Andrea Sella <
>> andrea.se...@radicalbit.io> wrote:
>>
>>> Hi Till,
>>>
>>> I've tried your suggestion (source-code
>>> <https://github.com/alkagin/alluxio-wordcount/>) and now it throws:
>>> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
>>> alluxio.hadoop.FileSystem not found.
>>> The core-site.xml has been set correctly and into the alluxio-wordcount
>>> jar is present alluxio-client. Do I need to specify the hadoop
>>> configuration via code or core-site.xml is enough?
>>>
>>> Thank you again,
>>> Andrea
>>>
>>> 2016-03-14 17:28 GMT+01:00 Till Rohrmann <trohrm...@apache.org>:
>>>
>>>> Hi Andrea,
>>>>
>>>> the problem won’t be netty-all but netty, I suspect. Flink is using
>>>> version 3.8 whereas alluxio-core-client uses version 3.2.2. I think
>>>> you have to exclude or shade this dependency away.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>>
>>>> On Mon, Mar 14, 2016 at 5:12 PM, Andrea Sella <
>>>> andrea.se...@radicalbit.io> wrote:
>>>>
>>>>> Hi Till,
>>>>> I tried to downgrade the Alluxio's netty version from 4.0.28.Final to
>>>>> 4.0.27.Final to align Flink and Alluxio dependencies. First of all, Flink
>>>>> 1.0.0 uses 4.0.27.Final, is it correct? Btw it doesn't work, same error as
>>>>> above.
>>>>>
>>>>> BR,
>>>>> Andrea
>>>>>
>>>>> 2016-03-14 15:30 GMT+01:00 Till Rohrmann <trohrm...@apache.org>:
>>>>>
>>>>>> Yes it seems as if you have a netty version conflict. Maybe the
>>>>>> alluxio-core-client.jar pulls in an incompatible netty version. Could you
>>>>>> check whether this is the case? But maybe you also have another
>>>>>> dependencies which pulls in a wrong netty version, since the Alluxio
>>>>>> documentation indicates that it works with Flink (but I cannot tell for
>>>>>> which version).
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>>>>> On Mon, Mar 14, 2016 at 3:18 PM, Andrea Sella <
>>>>>> andrea.se...@radicalbit.io> wrote:
>>>>>>
>>>>>>> Hi to all,
>>>>>>>
>>>>>>> I'm trying to integrate Alluxio and Apache Flink, I followed Running
>>>>>>> Flink on Alluxio
>>>>>>> <http://www.alluxio.org/documentation/en/Running-Flink-on-Alluxio.html>
>>>>>>> to
>>>>>>> setup Flink.
>>>>>>>
>>>>>>> I tested in local mode executing:
>>>>>>>
>>>>>>> bin/flink run ./examples/batch/WordCount.jar --input
>>>>>>> alluxio:///flink/README.txt
>>>>>>>
>>>>>>> But I've faced a TimeoutException, I attach the logs. It seems the
>>>>>>> trouble is due to netty dependencies-conflict.
>>>>>>>
>>>>>>> Thank you,
>>>>>>> Andrea
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
