[
https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269794#comment-15269794
]
Appy commented on HBASE-11625:
------------------------------
Here's how to reproduce.
I have uploaded two files:
- correct-hfile : Generated using hbase shell to write two rows to a new table
and flushing it.
- corrupted-header-hfile: Changed one character in 'correct-hfile' belonging to
block magic. Use {{cmp -b correct-hfile corrupted-header-hfile}} to see the
diff.
Since I messed with the file, it should report that data was corrupted but
instead it throws 'Invalid hfile block magic' exception (see below output).
The fix, as mentioned above, is to first do checksum and then make HFileBlock
instance.
{noformat}
appy@asharma-MBP ~/apache/hbase/bin (master) → ./hbase
org.apache.hadoop.hbase.io.hfile.HFile -p -f
file:///Users/appy/apache/hbase/correct-hfile
2016-05-03 15:47:37,332 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2016-05-03 15:47:37,534 INFO [main] hfile.CacheConfig: CacheConfig:disabled
K: 1/cf:1/1462311794785/Put/vlen=5/seqid=4 V: 11111
K: 1/cf:2/1462311803576/Put/vlen=6/seqid=5 V: 222222
Scanned kv count -> 2
appy@asharma-MBP ~/apache/hbase/bin (master) → ./hbase
org.apache.hadoop.hbase.io.hfile.HFile -p -f
file:///Users/appy/apache/hbase/corrupted-header-hfile
2016-05-03 15:47:43,932 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2016-05-03 15:47:44,089 INFO [main] hfile.CacheConfig: CacheConfig:disabled
2016-05-03 15:47:44,301 ERROR [main] hfile.HFilePrettyPrinter: Error reading
file:/Users/appy/apache/hbase/corrupted-header-hfile
java.io.IOException: Invalid HFile block magic: DcTABLK*
at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154)
at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock.<init>(HFileBlock.java:255)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1684)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1507)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:446)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:914)
at
org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.processFile(HFilePrettyPrinter.java:269)
at
org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.run(HFilePrettyPrinter.java:230)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.main(HFilePrettyPrinter.java:552)
at org.apache.hadoop.hbase.io.hfile.HFile.main(HFile.java:879)
{noformat}
> Reading datablock throws "Invalid HFile block magic" and can not switch to
> hdfs checksum
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-11625
> URL: https://issues.apache.org/jira/browse/HBASE-11625
> Project: HBase
> Issue Type: Bug
> Components: HFile
> Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3
> Reporter: qian wang
> Assignee: Pankaj Kumar
> Fix For: 2.0.0
>
> Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, HBASE-11625.patch,
> correct-hfile, corrupted-header-hfile
>
>
> when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it
> could happen file corruption but it only can switch to hdfs checksum
> inputstream till validateBlockChecksum(). If the datablock's header corrupted
> when b = new HFileBlock(),it throws the exception "Invalid HFile block magic"
> and the rpc call fail
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
