Re: [DISCUSS] Spark cannot identify the problem executor

2021-02-11 Thread attilapiros
Hi, There is an existing way to handle this situation. Those tasks will become zombie tasks [1] and they should not be counted into the tasks failures [2]. Even the shuffle blocks should be unregistered for the lost executor, although the lost executor might be already cached as a mapoutput in th

Re: dataFrame.na.fill() fails for column with dot

2021-02-11 Thread Amandeep Sharma
Adding note to the previous email. Test suite org.apache.spark.sql.catalyst.analysis.AnalysisSuite passed after the aforementioned changes. Regards, Amandeep Sharma On Tue, Feb 9, 2021 at 3:07 PM Amandeep Sharma wrote: > Hi guys, > Apologies for the long mail. > > I am running below code snipp