uot;better" than micro-batching -- they lose a lot of stuff that is possible
> in Spark, such as load balancing work dynamically across nodes, speculative
> execution for stragglers, scaling clusters up and down elastically, etc.
> Moreover, Spark itself could execute the current model
On Thu, Oct 20, 2016 at 12:07 AM Shivaram Venkataraman <
shiva...@eecs.berkeley.edu> wrote:
> At the AMPLab we've been working on a research project that looks at
> just the scheduling latencies and on techniques to get lower
> scheduling latency. It moves away from the micro-batch model, but
> re
I've been working on the Apache Beam Spark runner which is (in this
context) basically running a streaming model that focuses on event-time and
correctness with Spark, and as I see it (even in spark 1.6.x) the
micro-batches are really just added latency, which will work-out for some
users, and not
It should. Codegen uses the SparkConf in SparkEnv when instantiating a new
Serializer.
On Thu, Aug 4, 2016 at 6:14 PM Jacek Laskowski wrote:
> Hi Olivier,
>
> I don't know either, but am curious what you've tried already.
>
> Jacek
>
> On 3 Aug 2016 10:50 a.m., "Olivier Girardot" <
> o.girar...@
Should we backport https://github.com/apache/spark/pull/13424 to 1.6.2 ?
On Thu, Jun 16, 2016 at 9:02 AM andy petrella
wrote:
> +1 both too
> (for tomorrow lunchtime? ^^)
>
> On Thu, Jun 16, 2016 at 5:06 AM Raymond Honderdors <
> raymond.honderd...@sizmek.com> wrote:
>
>> +1 for both
>>
>> Get O
