Re: Hadoop s3 integration for Spark

2017-04-18 Thread Steve Loughran
On 14 Apr 2017, at 17:46, Afshin, Bardia mailto:bardia.afs...@capitalone.com>> wrote: Hello community. I’m considering consuming s3 objects via Hadoop via s3a protocol. The main purpose of this is to utilize Spark to access s3, and it seems like the only formal protocol / integration for doin

Hadoop s3 integration for Spark

2017-04-14 Thread Afshin, Bardia
Hello community. I’m considering consuming s3 objects via Hadoop via s3a protocol. The main purpose of this is to utilize Spark to access s3, and it seems like the only formal protocol / integration for doing so is Hadoop. The process that I am implementing is rather formal and straight forward