Would there be a way to force the 'old' data out? Because at this point
I'll have to restart the shell every couple of queries to get meaningful
timings which are comparable to spark submit .
On Jun 29, 2015 6:20 PM, "Mark Hamstra" wrote:
> No. He is collecting the results of the SQL query, not
Hi,
In the Spark UI, under “Show additional metrics”, there are two extra metrics
you can show
.1 Scheduler delay
.2 and Getting result time
When hovering “Scheduler Delay" it says (among other things):
…time to send task result from executor…
When hovering “Getting result time”:
Time that th
Hi,
Does anyone know of a Java/Scala library (not simply a HTTP library) for
interacting with Spark through its REST/HTTP API? My “problem” is that
interacting through REST induces a lot of work mapping the JSON to sensible
Spark/Scala objects.
So a simple example, I hope there is a library
The tuning documentations tells us to have 2-3 tasks per CPU core
> In general, we recommend 2-3 tasks per CPU core in your cluster.
I’m wondering how you’d actually accomplish this.
Setting spark.task.cpus to a fraction like 0.5 or 0.3 does not work.
Perhaps I’m misunderstanding, any advice i
f” parameter. Should these settings also be present in the
spark-1.2.0-bin-hadoop2.4.tgz distribution’s configs?
If, in order to be helped here, I need to present more logs etc, please let me
know.
Regards,
Hans van den Bogert
spark application to mesos executors and where this shared library
error comes from.
Hope someone can shed some light,
Thanks
On 08 Feb 2015, at 14:15, Hans van den Bogert wrote:
> Hi,
>
>
> I’m trying to get coarse mode to work under mesos(0.21.0), I thought this
> would be a tri
Bumping 1on1 conversation to mailinglist:
On 10 Feb 2015, at 13:24, Hans van den Bogert wrote:
>
> It’s self built, I can’t otherwise as I can’t install packages on the cluster
> here.
>
> The problem seems with libtool. When compiling Mesos on a host with apr-devel
> and
Hi all,
For my master thesis I will be characterising performance of two-level
schedulers like Mesos and after reading the paper:
https://www.cs.berkeley.edu/~alig/papers/mesos.pdf
where Spark is also introduced I am wondering how some experiments and results
came about.
If this is not the pl
