txwyy123 commented on issue #19929:
URL: https://github.com/apache/hudi/issues/19929#issuecomment-5648651239
### Full exception stack trace
```
HoodieException: Failed to instantiate Metadata table
Caused by: HoodieCompactionException: Could not compact
<table>/.hoodie/metadata
Caused by: HoodieIOException: unable to read next record from hfile
at
org.apache.hudi.io.storage.HoodieNativeAvroHFileReader$RecordIterator.hasNext(HoodieNativeAvroHFileReader.java:353)
Caused by: java.util.zip.ZipException: invalid block type
at
org.apache.hudi.io.storage.HoodieAirliftGzipDecompressor.decompress(...)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock.unpack(HFileBlock.java:199)
```
The exact `ZipException` message is **`invalid block type`** (not "Not in
GZIP format") — this points to the native reader's
`HoodieAirliftGzipDecompressor` failing to decompress a block that the
HBase-backed reader handles fine.
### Culprit file details
- **MDT partition:** `column_stats/`
- **Culprit HFile:** `col-stats-0000-0_0-383-1122919_<instant>.hfile` (a
`column_stats` partition base file)
- **HFile compression codec:** Default (GZIP) — the file was written by Hudi
1.0.2 writers with no write-time errors
- **HFile integrity:** Byte-valid on storage (verified via checksums); not a
storage corruption issue
### A/B result on the exact culprit file
| `_hoodie.hfile.use.native.reader` | Reader class | Result |
|---|---|---|
| `false` | `HoodieHBaseAvroHFileReader` | ✅ **SUCCESS — 9,115,795 records
read clean** |
| `true` | `HoodieNativeAvroHFileReader` | ❌ **FAILURE — `unable to read
next record from hfile` → `java.util.zip.ZipException: invalid block type`** |
Same bytes, same Avro schema (12 fields). The HBase reader fully iterates
the file; the native reader throws on the same block. This confirms the bug is
in the native reader's decompression path (`HoodieAirliftGzipDecompressor`),
not in the data.
### Crash context
The failure occurs during **MDT compaction** (a distributed Spark `collect`
on the `column_stats` partition). It crash-loops the writer because every write
attempt triggers MDT compaction, which re-reads the same block and throws.
### Workaround confirmed
`hoodie.metadata.reader.use.native.reader=false` in the writer config avoids
the crash with no functional regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
