[
https://issues.apache.org/jira/browse/CASSANDRA-16532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307187#comment-17307187
]
Adam Holmberg commented on CASSANDRA-16532:
-------------------------------------------
Unfortunately I am still seeing
[this|https://ci-cassandra.apache.org/job/Cassandra-trunk/362/testReport/junit/org.apache.cassandra.db/ScrubTest/testSkipScrubCorruptedCounterRowWithTool_compression/]
failure when looping on this branch. With a bit of instrumentation, I can see
that the scrubber is hitting this when it fails in this way:
{noformat}
Scrubbing
BigTableReader(path='/home/vagrant/cassandra/build/test/cassandra/data/Keyspace2/Counter1-949374d48be711eb9202f742d2eeefb4/na-1-big-Data.db')
(64.289KiB)
WARNING: Error reading row
org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
/home/vagrant/cassandra/build/test/cassandra/data/Keyspace2/Counter1-949374d48be711eb9202f742d2eeefb4/na-1-big-Data.db
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:138)
at
org.apache.cassandra.db.compaction.Scrubber$RowMergingSSTableIterator.hasNext(Scrubber.java:541)
at
org.apache.cassandra.db.compaction.Scrubber$OrderCheckerIterator.computeNext(Scrubber.java:654)
at
org.apache.cassandra.db.compaction.Scrubber$OrderCheckerIterator.computeNext(Scrubber.java:574)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
at
org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
at
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:205)
at
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
at
org.apache.cassandra.io.sstable.SSTableRewriter.tryAppend(SSTableRewriter.java:149)
at org.apache.cassandra.db.compaction.Scrubber.tryAppend(Scrubber.java:343)
at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:233)
at
org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:225)
at org.apache.cassandra.tools.ToolRunner.runClassAsTool(ToolRunner.java:82)
at org.apache.cassandra.tools.ToolRunner$2.get(ToolRunner.java:249)
at org.apache.cassandra.tools.ToolRunner$2.get(ToolRunner.java:245)
at
org.apache.cassandra.tools.ToolRunner.invokeSupplier(ToolRunner.java:305)
at org.apache.cassandra.tools.ToolRunner.invokeClass(ToolRunner.java:253)
at org.apache.cassandra.tools.ToolRunner.invokeClass(ToolRunner.java:235)
at
org.apache.cassandra.db.ScrubTest.testSkipScrubCorruptedCounterRowWithTool(ScrubTest.java:811)
Caused by: java.io.IOException: Error building row with data deserialized from
RandomAccessReader:CachingRebufferer:CompressedChunkReader.Mmap(/home/vagrant/cassandra/build/test/cassandra/data/Keyspace2/Counter1-949374d48be711eb9202f742d2eeefb4/na-1-big-Data.db
- SnappyCompressor, chunk length 16384, data length 65832)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:646)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeOne(UnfilteredSerializer.java:487)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:443)
at
org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:84)
at
org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:62)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:126)
Caused by: java.lang.AssertionError
at org.apache.cassandra.cache.ChunkCache$Buffer.buffer(ChunkCache.java:122)
at
org.apache.cassandra.io.util.RandomAccessReader.reBufferAt(RandomAccessReader.java:66)
at
org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:59)
at
org.apache.cassandra.io.util.RebufferingInputStream.read(RebufferingInputStream.java:90)
at
org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
at
org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:62)
at
org.apache.cassandra.db.marshal.ByteArrayAccessor.read(ByteArrayAccessor.java:101)
at
org.apache.cassandra.db.marshal.ByteArrayAccessor.read(ByteArrayAccessor.java:38)
at org.apache.cassandra.db.marshal.AbstractType.read(AbstractType.java:490)
at org.apache.cassandra.db.rows.Cell$Serializer.deserialize(Cell.java:268)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.readSimpleColumn(UnfilteredSerializer.java:655)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.lambda$deserializeRowBody$2(UnfilteredSerializer.java:620)
at org.apache.cassandra.utils.btree.BTree.applyValue(BTree.java:1294)
at org.apache.cassandra.utils.btree.BTree.applyLeaf(BTree.java:1302)
at org.apache.cassandra.utils.btree.BTree.apply(BTree.java:1317)
at org.apache.cassandra.utils.btree.BTree.apply(BTree.java:1343)
at org.apache.cassandra.db.Columns.apply(Columns.java:390)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:612)
{noformat}
It only happens when compression is enabled, and when the whole test class is
run (i.e.; not just one or a handful of methods).
> Fix flaky testSkipScrubCorruptedCounterRowWithTool
> --------------------------------------------------
>
> Key: CASSANDRA-16532
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16532
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
> Reporter: Berenguer Blasi
> Assignee: Berenguer Blasi
> Priority: Normal
> Fix For: 4.0-rc
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Fix flaky
> [testSkipScrubCorruptedCounterRowWithTool|https://ci-cassandra.apache.org/job/Cassandra-trunk/365/testReport/junit/org.apache.cassandra.db/ScrubTest/testSkipScrubCorruptedCounterRowWithTool_compression/]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
