Looking for answer to this.
Is it safe to delete the older files using
find . -type f -cmin +200 -name "shuffle*" -exec rm -rf {} \;
For a window duration of 2 hours how older files can we delete ?
Thanks.
On Sun, Feb 14, 2016 at 12:34 PM, Abhishek Anand
wrote:
> Hi All,
>
> Any ideas on thi
Hi All,
Any ideas on this one ?
The size of this directory keeps on growing.
I can see there are many files from a day earlier too.
Cheers !!
Abhi
On Tue, Jan 26, 2016 at 7:13 PM, Abhishek Anand
wrote:
> Hi Adrian,
>
> I am running spark in standalone mode.
>
> The spark version that I am us
Hi Adrian,
I am running spark in standalone mode.
The spark version that I am using is 1.4.0
Thanks,
Abhi
On Tue, Jan 26, 2016 at 4:10 PM, Adrian Bridgett
wrote:
> Hi Abhi - are you running on Mesos perchance?
>
> If so then with spark <1.6 you will be hitting
> https://issues.apache.org/jira
Hi Abhi - are you running on Mesos perchance?
If so then with spark <1.6 you will be hitting
https://issues.apache.org/jira/browse/SPARK-10975
With spark >= 1.6:
https://issues.apache.org/jira/browse/SPARK-12430
and also be aware of:
https://issues.apache.org/jira/browse/SPARK-12583
On 25/01/2
