Thanks all for the help on this. I've changed all my writes to
LOCAL_QUORUM, and same with reads. Under a constant load of doing
writes to a table and reads from the same table, I'm still getting the:
DEBUG [ReadRepairStage:372] 2020-12-14 09:36:09,002
ReadCallback.java:244 - Digest mismatch:
org.apache.cassandra.service.DigestMismatchException: Mismatch for key
DecoratedKey(-7287062361589376757,
44535f313034335f333332353839305f323032302d31322d31325430302d31392d33312e3330335a)
(054250ecd7170b1707ec36c6f1798ed0 vs 5752eec36bff050dd363b7803c500a95)
at
org.apache.cassandra.service.DigestResolver.compareResponses(DigestResolver.java:92)
~[apache-cassandra-3.11.9.jar:3.11.9]
at
org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:235)
~[apache-cassandra-3.11.9.jar:3.11.9]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[na:1.8.0_272]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_272]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
[apache-cassandra-3.11.9.jar:3.11.9]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_272]
Under load this happens a lot; several times a second on each of the
server nodes. I started with a new table and under light load, it
worked wonderfully - no issues. But under heavy load, it still occurs.
Is there a different setting?
Also, when this happens, I cannot query the table from presto as I then
get the familiar:
"Query 20201214_143949_00000_b3fnt failed: Cassandra timeout during read
query at consistency LOCAL_QUORUM (2 responses were required but only 1
replica responded)"
Changed presto to use ONE results in an error about 1 were required, but
only 1 responded.
Any ideas? Things to try? Thanks!
-Joe
On 12/3/2020 12:49 AM, Erick Ramirez wrote:
Thank you Steve - once I have the key, how do I get to a node?
Run this command to determine which replicas own the partition:
$ nodetool getendpoints <partition_key>
So if the propagation has not taken place and a node doesn't have
the data and is the first to 'be asked' the client will get no data?
That's correct. It will not return data it doesn't have when querying
with a consistency of ONE. There are limited cases where ONE is
applicable. In most cases, a strong consistency of LOCAL_QUORUM is
recommended to avoid the scenario you described. Cheers!
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
Virus-free. www.avg.com
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
