Hello Stephan & Nico, Here is the full stacktrace, its not much more than what I originally posted. I remember seeing an XMLInputFactory input error at one point, but I haven't seen that again. Is there any other information I can provide that will help resolve?
[flink-1.4.0] ./bin/flink run ~/streaming.jar Cluster configuration: Standalone cluster with JobManager at localhost/ 127.0.0.1:6123 Using address localhost:6123 to connect to JobManager. JobManager web interface address http://localhost:8082 Starting execution of program Submitting job with JobID: 022e4a310cd56ba4f7befc0286921663. Waiting for job completion. Connected to JobManager at Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-1321297259] with leader session id 00000000-0000-0000-0000-000000000000. 01/03/2018 11:21:53 Job execution switched to status RUNNING. 01/03/2018 11:21:53 Source: Kafka -> Sink: Unnamed(1/1) switched to SCHEDULED 01/03/2018 11:21:53 Source: Kafka -> Sink: Unnamed(1/1) switched to DEPLOYING 01/03/2018 11:21:53 Source: Kafka -> Sink: Unnamed(1/1) switched to RUNNING 01/03/2018 11:21:54 Source: Kafka -> Sink: Unnamed(1/1) switched to FAILED java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.internal.S3ErrorResponseHandler at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:363) at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:542) at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.createAmazonS3Client(PrestoS3FileSystem.java:639) at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:212) at org.apache.flink.fs.s3presto.S3FileSystemFactory.create(S3FileSystemFactory.java:132) at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:397) at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:320) at org.apache.flink.core.fs.Path.getFileSystem(Path.java:293) at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory.<init>(FsCheckpointStreamFactory.java:99) at org.apache.flink.runtime.state.filesystem.FsStateBackend.createStreamFactory(FsStateBackend.java:277) at org.apache.flink.streaming.runtime.tasks.StreamTask.createCheckpointStreamFactory(StreamTask.java:787) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:247) at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:694) at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:682) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) at java.lang.Thread.run(Thread.java:748) 01/03/2018 11:21:54 Job execution switched to status FAILING. java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.internal.S3ErrorResponseHandler at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:363) at org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:542) at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.createAmazonS3Client(PrestoS3FileSystem.java:639) at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:212) at org.apache.flink.fs.s3presto.S3FileSystemFactory.create(S3FileSystemFactory.java:132) at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:397) at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:320) at org.apache.flink.core.fs.Path.getFileSystem(Path.java:293) at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory.<init>(FsCheckpointStreamFactory.java:99) at org.apache.flink.runtime.state.filesystem.FsStateBackend.createStreamFactory(FsStateBackend.java:277) at org.apache.flink.streaming.runtime.tasks.StreamTask.createCheckpointStreamFactory(StreamTask.java:787) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:247) at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:694) at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:682) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) at java.lang.Thread.run(Thread.java:748) 01/03/2018 11:21:54 Job execution switched to status RESTARTING. On Wed, Jan 3, 2018 at 11:33 AM Stephan Ewen <se...@apache.org> wrote: > The error is not a the missing Class "S3ErrorResponseHandler", but the > initialization of that class. > > It could be missing classes that are statically referenced by the > "S3ErrorResponseHandler". > The ones I see are "XMLInputFactory" (part of Java itself, should always > be there) and "org.apache.commons.logging.Log" which should also be there. > There may be other classes loaded during initialization. Knowing more > about the exception would help, as we have not seen that error, yet. > > > On Wed, Jan 3, 2018 at 11:39 AM, Nico Kruber <n...@data-artisans.com> > wrote: > >> Hi Kyle, >> except for putting the jar into the lib/ folder and setting up >> credentials, nothing else should be required [1]. >> >> The S3ErrorResponseHandler class itself is in the jar, as you can see with >> jar tf flink-s3-fs-presto-1.4.0.jar | grep >> >> org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.internal.S3ErrorResponseHandler >> >> Therefore, the cause for this exception would be interesting (as Stephan >> suggested). >> >> >> Nico >> >> [1] >> >> https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/deployment/aws.html#shaded-hadooppresto-s3-file-systems-recommended >> >> On 03/01/18 10:22, Stephan Ewen wrote: >> > Hi Kyle! >> > >> > Is there more of the stack trace available, like an original exception >> > cause? >> > >> > Best, >> > Stephan >> > >> > >> > On Sun, Dec 31, 2017 at 5:10 PM, Kyle Hamlin <hamlin...@gmail.com >> > <mailto:hamlin...@gmail.com>> wrote: >> > >> > Hi, >> > >> > When testing Flink 1.4 locally the error below keeps getting thrown. >> > I've followed the setup by moving the flink-s3-fs-presto.jar from >> > the opt/ folder to the lib/ folder. Is there something additional I >> > need to do? >> > >> > java.lang.NoClassDefFoundError: Could not initialize class >> > >> >> org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.internal.S3ErrorResponseHandler >> > at >> > >> >> org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:363) >> > at >> > >> >> org.apache.flink.fs.s3presto.shaded.com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:542) >> > at >> > >> >> org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.createAmazonS3Client(PrestoS3FileSystem.java:639) >> > at >> > >> >> org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.PrestoS3FileSystem.initialize(PrestoS3FileSystem.java:212) >> > at >> > >> >> org.apache.flink.fs.s3presto.S3FileSystemFactory.create(S3FileSystemFactory.java:132) >> > at >> > >> >> org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:397) >> > at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:320) >> > at org.apache.flink.core.fs.Path.getFileSystem(Path.java:293) >> > at >> > >> >> org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory.<init>(FsCheckpointStreamFactory.java:99) >> > at >> > >> >> org.apache.flink.runtime.state.filesystem.FsStateBackend.createStreamFactory(FsStateBackend.java:277) >> > at >> > >> >> org.apache.flink.streaming.runtime.tasks.StreamTask.createCheckpointStreamFactory(StreamTask.java:787) >> > at >> > >> >> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:247) >> > at >> > >> >> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:694) >> > at >> > >> >> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:682) >> > at >> > >> >> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253) >> > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) >> > >> > >> >> >
