Hi Flavio,
It would be helpful, if we knew which class could not be found. In the
ClosureCleaner, can you change line 42 to include the class name in the
error message? Like in this example:
private static ClassReader getClassReader(Class<?> cls) {
String className = cls.getName().replaceFirst("^.*\\.", "") + ".class";
try {
return new ClassReader(cls.getResourceAsStream(className));
} catch (IOException e) {
throw new RuntimeException("Could not create ClassReader for
class " + cls.getName() + ":" + e);
}
}
Could it be that you're running an old job on the latest snapshot version?
This could cause class-related problems...
Cheers,
Max
On Wed, May 20, 2015 at 9:41 AM, Flavio Pompermaier <pomperma...@okkam.it>
wrote:
> Any insight about this..?
>
> On Tue, May 19, 2015 at 12:49 PM, Flavio Pompermaier <pomperma...@okkam.it
> > wrote:
>
>> Hi to all,
>>
>> I tried to run my job on a brand new Flink cluster (0.9-SNAPSHOT) from
>> the web client UI using the shading strategy of the quickstart example but
>> I get this exception:
>>
>> Caused by: java.lang.RuntimeException: Could not create ClassReader:
>> java.io.IOException: Class not found
>> at
>> org.apache.flink.api.java.ClosureCleaner.getClassReader(ClosureCleaner.java:42)
>> at
>> org.apache.flink.api.java.ClosureCleaner.cleanThis0(ClosureCleaner.java:67)
>> at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:54)
>>
>> It seems that it cannot find some kryo class..how do I fix this? this is
>> my shade plugin section of pom.xml:
>>
>>
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-shade-plugin</artifactId>
>> <version>1.4</version>
>> <executions>
>> <execution>
>> <phase>package</phase>
>> <goals>
>> <goal>shade</goal>
>> </goals>
>> <configuration>
>> <artifactSet>
>> <excludes>
>> <!-- This list contains all dependencies of flink-dist Everything
>> else will be packaged into the fat-jar -->
>> <exclude>org.apache.flink:flink-shaded-*</exclude>
>> <exclude>org.apache.flink:flink-core</exclude>
>> <exclude>org.apache.flink:flink-java</exclude>
>> <exclude>org.apache.flink:flink-scala</exclude>
>> <exclude>org.apache.flink:flink-runtime</exclude>
>> <exclude>org.apache.flink:flink-optimizer</exclude>
>> <exclude>org.apache.flink:flink-clients</exclude>
>> <exclude>org.apache.flink:flink-spargel</exclude>
>> <exclude>org.apache.flink:flink-avro</exclude>
>> <exclude>org.apache.flink:flink-java-examples</exclude>
>> <exclude>org.apache.flink:flink-scala-examples</exclude>
>> <exclude>org.apache.flink:flink-streaming-examples</exclude>
>> <exclude>org.apache.flink:flink-streaming-core</exclude>
>>
>> <!-- Also exclude very big transitive dependencies of Flink WARNING:
>> You have to remove these excludes if your code relies on other versions
>> of
>> these dependencies. -->
>> <exclude>org.scala-lang:scala-library</exclude>
>> <exclude>org.scala-lang:scala-compiler</exclude>
>> <exclude>org.scala-lang:scala-reflect</exclude>
>> <exclude>com.amazonaws:aws-java-sdk</exclude>
>> <exclude>com.typesafe.akka:akka-actor_*</exclude>
>> <exclude>com.typesafe.akka:akka-remote_*</exclude>
>> <exclude>com.typesafe.akka:akka-slf4j_*</exclude>
>> <exclude>io.netty:netty-all</exclude>
>> <exclude>io.netty:netty</exclude>
>> <exclude>org.eclipse.jetty:jetty-server</exclude>
>> <exclude>org.eclipse.jetty:jetty-continuation</exclude>
>> <exclude>org.eclipse.jetty:jetty-http</exclude>
>> <exclude>org.eclipse.jetty:jetty-io</exclude>
>> <exclude>org.eclipse.jetty:jetty-util</exclude>
>> <exclude>org.eclipse.jetty:jetty-security</exclude>
>> <exclude>org.eclipse.jetty:jetty-servlet</exclude>
>> <exclude>commons-fileupload:commons-fileupload</exclude>
>> <exclude>org.apache.avro:avro</exclude>
>> <exclude>commons-collections:commons-collections</exclude>
>> <exclude>org.codehaus.jackson:jackson-core-asl</exclude>
>> <exclude>org.codehaus.jackson:jackson-mapper-asl</exclude>
>> <exclude>com.thoughtworks.paranamer:paranamer</exclude>
>> <exclude>org.xerial.snappy:snappy-java</exclude>
>> <exclude>org.apache.commons:commons-compress</exclude>
>> <exclude>org.tukaani:xz</exclude>
>> <exclude>com.esotericsoftware.kryo:kryo</exclude>
>> <exclude>com.esotericsoftware.minlog:minlog</exclude>
>> <exclude>org.objenesis:objenesis</exclude>
>> <exclude>com.twitter:chill_*</exclude>
>> <exclude>com.twitter:chill-java</exclude>
>> <exclude>com.twitter:chill-avro_*</exclude>
>> <exclude>com.twitter:chill-bijection_*</exclude>
>> <exclude>com.twitter:bijection-core_*</exclude>
>> <exclude>com.twitter:bijection-avro_*</exclude>
>> <exclude>com.twitter:chill-protobuf</exclude>
>> <exclude>com.google.protobuf:protobuf-java</exclude>
>> <exclude>com.twitter:chill-thrift</exclude>
>> <exclude>org.apache.thrift:libthrift</exclude>
>> <exclude>commons-lang:commons-lang</exclude>
>> <exclude>junit:junit</exclude>
>> <exclude>de.javakaffee:kryo-serializers</exclude>
>> <exclude>joda-time:joda-time</exclude>
>> <exclude>org.apache.commons:commons-lang3</exclude>
>> <exclude>org.slf4j:slf4j-api</exclude>
>> <exclude>org.slf4j:slf4j-log4j12</exclude>
>> <exclude>log4j:log4j</exclude>
>> <exclude>org.apache.commons:commons-math</exclude>
>> <exclude>org.apache.sling:org.apache.sling.commons.json</exclude>
>> <exclude>commons-logging:commons-logging</exclude>
>> <exclude>org.apache.httpcomponents:httpclient</exclude>
>> <exclude>org.apache.httpcomponents:httpcore</exclude>
>> <exclude>commons-codec:commons-codec</exclude>
>> <exclude>com.fasterxml.jackson.core:jackson-core</exclude>
>> <exclude>com.fasterxml.jackson.core:jackson-databind</exclude>
>> <exclude>com.fasterxml.jackson.core:jackson-annotations</exclude>
>> <exclude>org.codehaus.jettison:jettison</exclude>
>> <exclude>stax:stax-api</exclude>
>> <exclude>com.typesafe:config</exclude>
>> <exclude>org.uncommons.maths:uncommons-maths</exclude>
>> <exclude>com.github.scopt:scopt_*</exclude>
>> <exclude>org.mortbay.jetty:servlet-api</exclude>
>> <exclude>commons-io:commons-io</exclude>
>> <exclude>commons-cli:commons-cli</exclude>
>> </excludes>
>> </artifactSet>
>> <filters>
>> <filter>
>> <artifact>org.apache.flink:*</artifact>
>> <excludes>
>> <exclude>org/apache/flink/shaded/**</exclude>
>> <exclude>web-docs/**</exclude>
>> </excludes>
>> </filter>
>> </filters>
>> <createDependencyReducedPom>false</createDependencyReducedPom>
>> <finalName>XXXX</finalName>
>> <transformers>
>> <!-- add Main-Class to manifest file -->
>> <transformer
>>
>> implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
>> <manifestEntries>
>> <Main-Class>XXX</Main-Class>
>> </manifestEntries>
>> </transformer>
>> </transformers>
>> </configuration>
>> </execution>
>> </executions>
>> </plugin>
>>
>
>
