Once you configured a custom file system in Hadoop it can be used by Spark out of the box. Depending what you implement in the custom file system you may think about side effects to any application including spark (memory consumption etc).
> On 21 Nov 2016, at 18:26, Samy Dindane <s...@dindane.com> wrote: > > Hi, > > I'd like to extend the file:// file system and add some custom logic to the > API that lists files. > I think I need to extend FileSystem or LocalFileSystem from > org.apache.hadoop.fs, but I am not sure how to go about it exactly. > > How to write a custom file system and make it usable by Spark? > > Thank you, > > Samy > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
