Looks like this is already solved in
https://issues.apache.org/jira/browse/SPARK-6330
On Mon, Mar 16, 2015 at 6:43 PM, Cheng Lian wrote:
> Oh sorry, I misread your question. I thought you were trying something
> like parquetFile(“s3n://file1,hdfs://file2”). Yeah, it’s a valid bug.
> Thanks for
Oh sorry, I misread your question. I thought you were trying something
like |parquetFile(“s3n://file1,hdfs://file2”)|. Yeah, it’s a valid bug.
Thanks for opening the JIRA ticket and the PR!
Cheng
On 3/16/15 6:39 PM, Cheng Lian wrote:
Hi Pei-Lun,
We intentionally disallowed passing multiple
Hi Pei-Lun,
We intentionally disallowed passing multiple comma separated paths in
1.3.0. One of the reason is that users report that this fail when a file
path contain an actual comma in it. In your case, you may do something
like this:
|val s3nDF = parquetFile("s3n://...")
val hdfsDF =
