Hi,
I'm looking for a way to get an aggregated view of data locality across a
spark job/stage.
I was sure that this existed but I can't find it now.
Basically a quick summary on how many tasks were PROCESS_LOCAL vs,
NODE_LOCAL etc.
Is there a way to do this or a Jira to track (I couldn't find an
i'm aware and will be looking in to this later today.
see:
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7/6043/console
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
(We have two PRs to patch it up anyway already)
On Mon, Jun 24, 2019 at 11:39 AM shane knapp wrote:
>
> i'm aware and will be looking in to this later today.
>
> see:
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7/6043/console
>
> --
> Shane Knapp
> UC Berkeley EECS
ah, ok. thanks for letting me know. :)
On Mon, Jun 24, 2019 at 9:39 AM Sean Owen wrote:
> (We have two PRs to patch it up anyway already)
>
> On Mon, Jun 24, 2019 at 11:39 AM shane knapp wrote:
> >
> > i'm aware and will be looking in to this later today.
> >
> > see:
> >
> https://amplab.cs.
Hi
In other to address this issue, as well as other use cases as virtual kubelet,
I’ve created this jira ticket.
https://issues.apache.org/jira/browse/SPARK-28149
From: Jose Luis Pedrosa
Date: Tuesday 18 June 2019 at 16:38
To: "Prudhvi Chennuru (CONT)"
Cc: Olivier Girardot , Li Gao
, dev ,
Hi All,
What Java version should I use to build Spark on Ubuntu? What are the
instructions on installing it on Ubuntu?
Official doc on this is missing this info:
https://spark.apache.org/docs/latest/building-spark.html
If I use default JDK, I get a build error Googling which shows that I need
to
"The Maven-based build is the build of reference for Apache Spark.
Building Spark using Maven requires Maven 3.5.4 and Java 8."
It doesn't depend on a particular version of Java 8. Installing it is
platform-dependent.
On Mon, Jun 24, 2019 at 6:43 PM Valeriy Trofimov
wrote:
>
> Hi All,
>
> What Ja
Well it used to work, but I can't say for sure when it failed (I don't use
this link every so often :) )
It works now, thanks !
Le mar. 18 juin 2019 à 15:29, Sean Owen a écrit :
> Huh, I don't know how long that's been a bug, but the JS that creates
> the filename with .replace doesn't seem to h
