milenkovicm opened a new issue, #1219: URL: https://github.com/apache/datafusion-ballista/issues/1219
**Describe the bug** If the `pull-staged` strategy is selected job data won't be cleared after the job is finished, leaving data hanging on the executes. This can lead to piling old shuffle files on the executor. One way to prevent is to set executors to cleanup data more aggressively. **To Reproduce** just run default ballista cluster setup **Expected behavior** Shuffle files to be removed when job finishes or when there is no need for them **Additional context** - `push-based` strategy works as expected - It might be related to #1175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
