Dear Spark Users, We have run into an issue where with spark 3.3.2 using auto scaling with STS is working fine, but with 3.4.2 or 3.5.2 executors are being left behind and not scaling down. Driver makes a call to remove the executor but some (not all) executors never get removed.
Has anyone else noticed this or aware of any reported issues? Any help will be greatly appreciated. Regards Jay
