sarutak opened a new pull request, #54660: URL: https://github.com/apache/spark/pull/54660
### What changes were proposed in this pull request? This PR proposes to add more tests for SHS multiple log directories feature added in SPARK-55793 (#54575). New tests include: - **directory removed while SHS is running** — verifies that removing a log directory at runtime does not crash the scan and apps from remaining directories are still listed - **directory does not exist at startup but created later** — verifies that a directory that doesn't exist at startup is picked up on subsequent scans (monthly directory scenario) - **directory temporarily inaccessible then recovers** — verifies that apps reappear after a temporarily inaccessible directory is restored - **all directories inaccessible does not crash** — verifies graceful handling when all configured directories become unavailable - **config with empty entries between commas** — verifies that empty entries in `spark.history.fs.logDirectory` (e.g., `dir1,,dir2`) are handled correctly - **logDirectory.names count mismatch falls back to full paths** — verifies that when the number of names doesn't match the number of directories, display names fall back to full paths ### Why are the changes needed? For better test coverage. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Confirmed that all new tests passed. ``` $ build/sbt 'testOnly org.apache.spark.deploy.history.RocksDBBackendFsHistoryProviderSuite' ``` ### Was this patch authored or co-authored using generative AI tooling? Kiro CLI / Opus 4.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
