siewcapital opened a new pull request, #63040: URL: https://github.com/apache/airflow/pull/63040
### What this PR does `S3Hook.sync_to_local_dir(..., delete_stale=True)` currently checks only the top-level entries of `local_dir` when deleting stale files. As a result, stale DAG files inside nested folders are never removed. This PR makes stale cleanup recursive by traversing all descendants of `local_dir` (deepest path first), so stale nested files can be deleted and then their empty parent directories removed. ### Why this change This addresses #62622 where `S3DagBundle` keeps stale DAGs in subfolders because `Path.iterdir()` is not recursive. ### Tests Updated `test_sync_to_local_dir_behaviour` to cover nested stale cleanup: - creates `stale/nested/dag_stale.py` locally (not in S3) - verifies stale nested file is deleted - verifies both `stale/nested` and `stale` empty directories are deleted I also validated syntax compilation for the touched files locally. Closes #62622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
