Hi Yash,
Thank you for the response.
Sorry it was not at distinct but it was at a join stage.
It was a self join. There were no errors and the jobs were stuck at the
step for a around 7 hrs, the last message that came through was .
*ShuffleBlockFetcherIterator: Started 4 remote fetches*
Thanks,
Ra
Hi Ramesh,
Could you share some logs please? pastebin ? dag view ?
Did you check for GC pauses if any.
On Thu, 6 Apr 2017 at 21:55 Ramesh Krishnan wrote:
> I have a use case of distinct on a dataframe. When i run the application
> is getting stuck at LINE *ShuffleBlockFetcherIterator: Started 4
I have a use case of distinct on a dataframe. When i run the application is
getting stuck at LINE *ShuffleBlockFetcherIterator: Started 4 remote
fetches *forever.
Can someone help .
Thanks
Ramesh
