bhavikpatel9977 commented on a change in pull request #3411: ZEPPELIN-4272:
Zeppelin fails to compile when hadoop3 is enabled
URL: https://github.com/apache/zeppelin/pull/3411#discussion_r308610519
##########
File path:
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java
##########
@@ -60,16 +59,25 @@
public FileSystemStorage(ZeppelinConfiguration zConf, String path) throws
IOException {
this.zConf = zConf;
this.hadoopConf = new Configuration();
+ URI zepConfigURI;
+
+ zepConfigURI = getURI(path, "Failed to get URI path");
// disable checksum for local file system. because interpreter.json may be
updated by
// non-hadoop filesystem api
// disable caching for file:// scheme to avoid getting LocalFS which does
CRC checks
+
this.hadoopConf.setBoolean("fs.file.impl.disable.cache", true);
- this.hadoopConf.set("fs.file.impl", RawLocalFileSystem.class.getName());
Review comment:
We have set **zeppelin.notebook.dir** and **zeppelin.config.fs.dir** to s3a
FileSystemStorage.
If we don't remove this line then it throws exception
``` WARN [2019-07-29 22:47:49,335] ({main} ContextHandler.java[log]:2359) -
unavailable
MultiException stack 1 of 1
java.lang.ClassCastException: org.apache.hadoop.fs.RawLocalFileSystem cannot
be cast to org.apache.hadoop.fs.LocalFileSystem
at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:447)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:301)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:378)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.createTmpFileForWrite(LocalDirAllocator.java:461)
at
org.apache.hadoop.fs.LocalDirAllocator.createTmpFileForWrite(LocalDirAllocator.java:200)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.createTmpFileForWrite(S3AFileSystem.java:753)
at
org.apache.hadoop.fs.s3a.S3ADataBlocks$DiskBlockFactory.create(S3ADataBlocks.java:811)
at
org.apache.hadoop.fs.s3a.S3ABlockOutputStream.createBlockIfNeeded(S3ABlockOutputStream.java:190)
at
org.apache.hadoop.fs.s3a.S3ABlockOutputStream.<init>(S3ABlockOutputStream.java:168)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:1044)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1130)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1110)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:999)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:987)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
