peterxcli commented on issue #5023: URL: https://github.com/apache/datafusion-comet/issues/5023#issuecomment-5071805110
Issue https://github.com/apache/datafusion-comet/issues/5023 is an upstream **libhdfs `JNIEnv` use-after-free**, not a Parquet failure or a regression from https://github.com/apache/datafusion-comet/pull/5014. ### Evidence - I downloaded the failing job’s `hs_err_pid22862.log` and exact `native-lib-macos` artifact, then symbolized it: - `pc = 0` - `lr = hdfsThreadDestructor + 80` - `x8 = 0` - Disassembly shows `hdfsThreadDestructor` loads the JNI `GetJavaVM` function pointer into `x8`, then executes `blr x8`. The stale `JNIEnv` contains a null function entry, producing the jump to address zero. - The vulnerable call is exactly [`(*env)->GetJavaVM`](<~/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/hdfs-sys-0.3.0/libhdfs/hdfs_3_3/os/posix/thread_local_storage.c:57>). Its null checks cannot detect a freed-but-non-null `JNIEnv`. - This precisely matches Apache [HDFS-16021: heap-use-after-free in hdfsThreadDestructor](https://issues.apache.org/jira/browse/HDFS-16021)and the earlier [HDFS-14488](https://issues.apache.org/jira/browse/HDFS-14488). `HDFS-16021` remains open without a fix version, and even current Hadoop trunk still contains the vulnerable [destructor implementation](https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/os/posix/thread_local_storage.c#L46-L58). The Comet path is: `fake:// → libhdfs scheme → OpenDAL HDFS → hdrs → hdfs-sys 0.3.0 → vendored libhdfs` The test explicitly registers `fake` as a libhdfs scheme in [`ParquetReadFromFakeHadoopFsSuite.scala`](</Users/lixucheng/Documents/oss/apache/datafusion-comet/spark/src/test/scala/org/apache/comet/parquet/ParquetReadFromFakeHadoopFsSuite.scala:44>). Native routing occurs in [`is_hdfs_scheme`](</Users/lixucheng/Documents/oss/apache/datafusion-comet/native/core/src/parquet/parquet_support.rs:426>) and [`create_hdfs_object_store`](</Users/lixucheng/Documents/oss/apache/datafusion-comet/native/core/src/parquet/parquet_support.rs:458>). macOS explicitly enables vendored `hdrs` in [`Cargo.toml`](</Users/lixucheng/Documents/oss/apache/datafusion-comet/native/core/Cargo.toml:83>), while [`Cargo.lock`](</Users/lixucheng/Documents/oss/apache/datafusion-comet/native/Cargo.lock:3275>) pins `hdfs-sys 0.3.0`. ### Recommended fix Carry or upstream the [HDFS-16021 patch](https://issues.apache.org/jira/secure/attachment/13025245/fix-hdfsThreadDestructor.patch): record whether libhdfs itself attached the thread, and only call `GetJavaVM`/`DetachCurrentThread` when libhdfs owns that attachment. No released `hdfs-sys` upgrade currently includes it. _Assist by LLM_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
