[
https://issues.apache.org/jira/browse/CASSANDRA-4316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13559979#comment-13559979
]
Yuki Morishita commented on CASSANDRA-4316:
-------------------------------------------
If we are compaction 32 SSTables at once, we are currently throttling every 32
rows read. If we are using LazilyCompactedRow for wide rows, the row is read
twice to write out compacted row (for now). And I don't know if that "N = 1MB /
average row size" above fits well for various row sizes. "1MB" sounds like the
same as 1000 in current "% 1000" approach.
So instead pushing down throttling at SSTableScanner level, we can throttle
more accurately.
Maybe we can go further and use RateLimiter in RandomAccessReader for more
accuracy.
> Compaction Throttle too bursty with large rows
> ----------------------------------------------
>
> Key: CASSANDRA-4316
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4316
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.8.0
> Reporter: Wayne Lewis
> Assignee: Yuki Morishita
> Fix For: 1.2.1
>
> Attachments: 4316-1.2.txt, 4316-1.2-v2.txt
>
>
> In org.apache.cassandra.db.compaction.CompactionIterable the check for
> compaction throttling occurs once every 1000 rows. In our workload this is
> much too large as we have many large rows (16 - 100 MB).
> With a 100 MB row, about 100 GB is read (and possibly written) before the
> compaction throttle sleeps. This causes bursts of essentially unthrottled
> compaction IO followed by a long sleep which yields inconsistence performance
> and high error rates during the bursts.
> We applied a workaround to check throttle every row which solved our
> performance and error issues:
> line 116 in org.apache.cassandra.db.compaction.CompactionIterable:
> if ((row++ % 1000) == 0)
> replaced with
> if ((row++ % 1) == 0)
> I think the better solution is to calculate how often throttle should be
> checked based on the throttle rate to apply sleeps more consistently. E.g. if
> 16MB/sec is the limit then check for sleep after every 16MB is read so sleeps
> are spaced out about every second.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
