Thank you!
Clocks were out of sync; chronyd wasn't chrony'ding.
Going so much faster now! Cheers.
-Joe
On 9/15/2021 4:07 PM, Bowen Song wrote:
Well, the log says cross node timeout, latency a bit over 44 seconds.
Here's a few most likely causes:
1. The clocks are not in sync - please check
Well, the log says cross node timeout, latency a bit over 44 seconds.
Here's a few most likely causes:
1. The clocks are not in sync - please check the time on each server,
and ensure NTP client is running on all Cassandra servers
2. Long stop the world GC pauses - please check the GC logs an
Thank you Erick - looking through all the logs on the nodes I found this:
INFO [CompactionExecutor:17551] 2021-09-15 15:13:20,524
CompactionTask.java:245 - Compacted
(fb0cdca0-1658-11ec-9098-dd70c3a3487a) 4 sstables to
[/data/7/cassandra/data/doc/fieldcounts-03b67080ada111ebade9fdc1d34336d3/n
Check the logs on the Cassandra servers first. Many different things can
cause the same result, and you will have to dig in deeper to discover
the true cause.
On 14/09/2021 23:55, Joe Obernberger wrote:
I'm getting a lot of the following errors during ingest of data:
com.datastax.oss.driver.a
The obvious conclusion is to say that the nodes can't keep up so it would
be interesting to know how often you're issuing the counter updates. Also,
how are the commit log disks performing on the nodes? If you have
monitoring in place, check the IO stats/metrics. And finally, review the
logs on the
I'm getting a lot of the following errors during ingest of data:
com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException:
Cassandra timeout during COUNTER write query at consistency ONE (1
replica were required but only 0 acknowledged the write)
at
com.datastax.oss.driver.a
