Also I see
WARN [ReadRepairStage:341] 2018-12-31 17:57:58,594 DataResolver.java:507 -
Encountered an oversized (37264537/16777216) read repair mutation for table
for about several hours (5-7) after cluster restart, next it disappeared.
On Tuesday, January 1, 2019 1:10 PM, Vlad <qa23d-...@yahoo.com.INVALID>
wrote:
Actually what happened is that Cassandra instances was upgraded to bigger size
one by one, with downtime about one-two minutes each. There are logs
INFO [HintsDispatcher:3] 2018-12-31 11:23:48,305
HintsDispatchExecutor.java:282 - Finished hinted handoff of file
d2c7bb82-3d7a-43b2-8791-ef9c7c02b2c1-1546182664318-1.hints to endpoint
/10.123.123.123: d2c7bb82-3d7a-43b2-8791-ef9c7c02b2c1
And from now on there is lot of Digest Mismatch exceptions in Cassandra log.
What's going on?
On Tuesday, January 1, 2019 10:18 AM, Vlad <qa23d-...@yahoo.com.INVALID>
wrote:
Hi All and Happy New Year!!!
This year started with Cassandra 3.11.3 sometimes forces level ALL despite
query level LOCAL_QUORUM (actually there is only one DC) and it fails with
timeout.
As far as I understand, it can be caused by read repair attempts (we see
"DigestMismatch" errors in Cassandra log), but table has no read repair
configured:
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
Any suggestions?
Thanks.
