Re: CSV escaping not working

2016-10-28 Thread Daniel Barclay
rowse/CSV-135> From: Koert Kuipers mailto:ko...@tresata.com>> Date: Thursday, October 27, 2016 at 12:49 PM To: "Jain, Nishit" mailto:nja...@underarmour.com>> Cc: "user@spark.apache.org <mailto:user@spark.apache.org>" mailto:user@spark.apac

Re: CSV escaping not working

2016-10-27 Thread Koert Kuipers
e good to see something > documented. > This may be the cause of the issue?: https://issues.apache. > org/jira/browse/CSV-135 > > From: Koert Kuipers > Date: Thursday, October 27, 2016 at 12:49 PM > > To: "Jain, Nishit" > Cc: "user@spark.apache.org"

Re: CSV escaping not working

2016-10-27 Thread Jain, Nishit
>> Date: Thursday, October 27, 2016 at 12:49 PM To: "Jain, Nishit" mailto:nja...@underarmour.com>> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: CSV escaping not working well my expectation would be

Re: CSV escaping not working

2016-10-27 Thread Koert Kuipers
uld escaping not work without quotes? > > From: Koert Kuipers > Date: Thursday, October 27, 2016 at 12:40 PM > To: "Jain, Nishit" > Cc: "user@spark.apache.org" > Subject: Re: CSV escaping not working > > that is what i would expect: escaping only work

Re: CSV escaping not working

2016-10-27 Thread Jain, Nishit
t;" mailto:user@spark.apache.org>> Subject: Re: CSV escaping not working that is what i would expect: escaping only works if quoted On Thu, Oct 27, 2016 at 1:24 PM, Jain, Nishit mailto:nja...@underarmour.com>> wrote: Interesting finding: Escaping works if data is quoted b

Re: CSV escaping not working

2016-10-27 Thread Koert Kuipers
o: "user@spark.apache.org" > Subject: CSV escaping not working > > I am using spark-core version 2.0.1 with Scala 2.11. I have simple code to > read a csv file which has \ escapes. > > val myDA = spark.read > .option("quote",null) > .schema(mySchema)

Re: CSV escaping not working

2016-10-27 Thread Jain, Nishit
Interesting finding: Escaping works if data is quoted but not otherwise. From: "Jain, Nishit" mailto:nja...@underarmour.com>> Date: Thursday, October 27, 2016 at 10:54 AM To: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>

CSV escaping not working

2016-10-27 Thread Jain, Nishit
I am using spark-core version 2.0.1 with Scala 2.11. I have simple code to read a csv file which has \ escapes. val myDA = spark.read .option("quote",null) .schema(mySchema) .csv(filePath) As per documentation \ is default escape for csv reader. But it does not work. Spark is rea