OK. We have a long running streaming job. I was thinking that may be we
should have a cron to clear files that are older than 2 days. What would be
an appropriate way to do that?
On Wed, Nov 18, 2015 at 7:43 PM, Ted Yu wrote:
> Have you seen SPARK-5836 ?
> Note TD's comment at the end.
>
> Cheer
Have you seen SPARK-5836 ?
Note TD's comment at the end.
Cheers
On Wed, Nov 18, 2015 at 7:28 PM, swetha wrote:
> Hi,
>
> We have a lot of temp files that gets created due to shuffles caused by
> group by. How to clear the files that gets created due to intermediate
> operations in group by?
>
>
