Yicong-Huang opened a new pull request, #6201: URL: https://github.com/apache/texera/pull/6201
### What changes were proposed in this PR? Upgrade `org.apache.hadoop:hadoop-common` to 3.4.3 in both places it is pinned (`common/workflow-core/build.sbt` was at 3.3.1, `amber/build.sbt` at 3.3.3), and remove `hadoop-mapreduce-client-core` from workflow-core. This resolves 4 open Dependabot alerts on `hadoop-common`: - [#656](https://github.com/apache/texera/security/dependabot/656) CVE-2022-25168 (critical, argument injection in `FileUtil.unTar`, fixed in 3.3.3) - [#732](https://github.com/apache/texera/security/dependabot/732) CVE-2022-26612 (critical, fixed in 3.3.3) - [#654](https://github.com/apache/texera/security/dependabot/654) CVE-2021-37404 (critical, heap overflow in libhdfs, fixed in 3.3.2) - [#700](https://github.com/apache/texera/security/dependabot/700) CVE-2024-23454 (low, fixed in 3.4.0) The dependency itself cannot be removed: Iceberg's `HadoopCatalog`/`HadoopFileIO` (used by `IcebergUtil` in workflow-core) and `HDFSRecordStorage` (amber) require hadoop-common at runtime. `hadoop-mapreduce-client-core`, however, has no compile-time or runtime references anywhere in the codebase (no `org.apache.hadoop.mapreduce` usages), so it is dropped instead of upgraded. ### Any related issues, documentation, discussions? Dependabot alert https://github.com/apache/texera/security/dependabot/656 (also 732, 654, 700). ### How was this PR tested? Tested with existing test cases: - `sbt WorkflowCore/test`: 458 tests passed, 0 failed. (5 suites — S3/LakeFS/LargeBinary specs — abort because they need external MinIO/LakeFS services; verified they abort identically on unmodified `main`, unrelated to this change.) - `sbt WorkflowExecutionService/Test/compile` and `WorkflowExecutionService/testOnly ...SequentialRecordStorageSpec` (covers the `HDFSRecordStorage` code path): 9 tests passed. - `IcebergUtilSpec` and `DocumentFactorySpec` (Iceberg + hadoop `Configuration` usage) pass, confirming nothing needed `hadoop-mapreduce-client-core`. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Fable 5) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
