Hi,
Can anyone please point me to the right class(es) where I can start digging
in ?
That would be super helpful too.
Thanks,
Aviral Agarwal
On Fri, Mar 24, 2017 at 3:58 PM, Aviral Agarwal
wrote:
> Hi ,
> Can you please point me on how to resolve the expression ?
> I was looking into LogicalPla
1) I think this depends on individual case by case jira. I haven't looked in
detail at spark-14649 seems much larger although more the way I think we want
to go. While SPARK-13669 seems less risky and easily configurable.
2) I don't know whether it needs an entire rewrite but I think there need
Hi all,
I am trying to trap UI kill event of a spark application from driver.
Some how the exception thrown is not propagated to the driver main
program. See for example using spark-shell below.
Is there a way to get hold of this event and shutdown the driver program?
Regards,
Noorul
spark@spa
(1) I'm pretty hesitant to merge these larger changes, even if they're
feature flagged, because:
(a) For some of these changes, it's not obvious that they'll always
improve performance. e.g., for SPARK-14649, it's possible that the tasks
that got re-started (and temporarily are running in two pl
Hi
I have a job which saves a dataframe as parquet file to s3.
The built a jar using your repository https://github.com/rdblue/s3committer.
I added the following config in the to the Spark Session
config("spark.hadoop.spark.sql.parquet.output.committer.class",
"com.netflix.bdp.s3.S3Partitioned
Following up on this with a renewed plea to file JIRAs when you see flaky
tests. I did a quick skim of the PR builder, and there were 17 times in
the last week when a flaky test led to a Jenkins failure, and someone
re-ran the tests without filing (or updating) a JIRA (my apologies to
anyone who w
