Keith Turner created HADOOP-16080:
-------------------------------------
Summary: hadoop-aws does not work with hadoop-client-api
Key: HADOOP-16080
URL: https://issues.apache.org/jira/browse/HADOOP-16080
Project: Hadoop Common
Issue Type: Improvement
Components: fs/s3
Affects Versions: 3.1.1
Reporter: Keith Turner
I attempted to use Accumulo and S3a with the following jars on the classpath.
* hadoop-client-api-3.1.1.jar
* hadoop-client-runtime-3.1.1.jar
* hadoop-aws-3.1.1.jar
This failed with the following exception.
{noformat}
Exception in thread "init" java.lang.NoSuchMethodError:
org.apache.hadoop.util.SemaphoredDelegatingExecutor.<init>(Lcom/google/common/util/concurrent/ListeningExecutorService;IZ)V
at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:769)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1169)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1149)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1108)
at org.apache.hadoop.fs.FileSystem.createNewFile(FileSystem.java:1413)
at
org.apache.accumulo.server.fs.VolumeManagerImpl.createNewFile(VolumeManagerImpl.java:184)
at org.apache.accumulo.server.init.Initialize.initDirs(Initialize.java:479)
at
org.apache.accumulo.server.init.Initialize.initFileSystem(Initialize.java:487)
at
org.apache.accumulo.server.init.Initialize.initialize(Initialize.java:370)
at org.apache.accumulo.server.init.Initialize.doInit(Initialize.java:348)
at org.apache.accumulo.server.init.Initialize.execute(Initialize.java:967)
at org.apache.accumulo.start.Main.lambda$execKeyword$0(Main.java:129)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The problem is that {{S3AFileSystem.create()}} looks for
{{SemaphoredDelegatingExecutor(com.google.common.util.concurrent.ListeningExecutorService)}}
which does not exist in hadoop-client-api-3.1.1.jar. What does exist is
{{SemaphoredDelegatingExecutor(org.apache.hadoop.shaded.com.google.common.util.concurrent.ListeningExecutorService)}}.
To work around this issue I created a version of hadoop-aws-3.1.1.jar that
relocated references to Guava.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
