Re: Strange exception while reading Parquet files

Hi, Does this issue also occur in v1.6.1 and v2.0-preview? // maropu On Thu, Jun 9, 2016 at 8:15 AM, Rostyslav Sotnychenko < r.sotnyche...@gmail.com> wrote: > Hello! > > I have faced a very strange exception (stack-trace in the end of this > email) while trying to read Parquet file using Hive C

Re: Add Caller Context in Spark

Yes, it is a string. Jira SPARK-15857 is created. Thanks, WQ On Thu, Jun 9, 2016 at 4:40 PM, Reynold Xin wrote: > Is this just to set some string? That makes sense. One thing you would > need to make sure is that Spark should still work outsid

Re: Add Caller Context in Spark

Is this just to set some string? That makes sense. One thing you would need to make sure is that Spark should still work outside of Hadoop, and also in older versions of Hadoop. On Thu, Jun 9, 2016 at 4:37 PM, Weiqing Yang wrote: > Hi, > > Hadoop has implemented a feature of log tracing – caller

Add Caller Context in Spark

Hi, Hadoop has implemented a feature of log tracing – caller context (Jira: HDFS-9184 and YARN-4349 ). The motivation is to better diagnose and understand how specific applications impacting parts of

Re: JIRA SPARK-2984

I'd do some searching and see if there is a JIRA related to this problem on s3 and if you don't find one go ahead and make one. Even if it is an intrinsic problem with s3 (and I'm not super sure since I'm just reading this on mobile) - it would maybe be a good thing for us to document. On Thursday

Re: JIRA SPARK-2984

I think your error could possibly be different - looking at the original JIRA the issue was happening on HDFS and you seem to be experiencing the issue on s3n, and while I don't have full view of the problem I could see this being s3 specific (read-after-write on s3 is trickier than read-after-writ

Strange exception while reading Parquet files

Hello! I have faced a very strange exception (stack-trace in the end of this email) while trying to read Parquet file using Hive Context from Spark 1.3.1, Hive 0.13. This issue appears only on YARN (standalone and local are working fine) and only when HiveContext is used (from SqlContext everythi

Re: Spark 2.0.0 preview docs uploaded

Available but mostly as JIRA output: https://spark.apache.org/news/spark-2.0.0-preview.html On Thu, Jun 9, 2016 at 7:33 AM, Pete Robbins wrote: > It would be nice to have a "what's new in 2.0.0" equivalent to > https://spark.apache.org/releases/spark-release-1-6-0.html available or am I > just mi