Yes, the data is stored in driver memory.
Mehdi Ben Haj Abbes 于2016年1月29日星期五 18:13写道：
> Thanks Terry for the quick answer.
>
> I did not tried it. Lets say I will increase the value to 2, what
> side effect should I expect. In fact in the explanation of the property "How
> many finished batc
Thanks Terry for the quick answer.
I did not tried it. Lets say I will increase the value to 2, what side
effect should I expect. In fact in the explanation of the property "How
many finished batches the Spark UI and status APIs remember before garbage
collecting." So the data is stored in me
Hi Mehdi,
Do you try a larger value of "spark.streaming.ui.retainedBatches"(default
is 1000)?
Regards,
- Terry
On Fri, Jan 29, 2016 at 5:45 PM, Mehdi Ben Haj Abbes
wrote:
> Hi folks,
>
> I have a streaming job running for more than 24 hours. It seems that there
> is a limit on the number of th
Hi folks,
I have a streaming job running for more than 24 hours. It seems that there
is a limit on the number of the batches displayed in the Streaming Statics
visualization screen. For example if I would launch a job Friday I will not
be able to have the statistics for what happened during Saturd
