I just noticed that swappiness is set to 60 at the slow site and 1 at the
other site. I am going to work with the system administrators to change
this as soon as possible.
Thanks,
On Thu, Mar 16, 2023 at 9:34 AM Vincent Russell
wrote:
> Thank you Dave. I didn't take look at the slow sync c
Thank you Dave. I didn't take look at the slow sync cost message when I
shut those nodes down. I just monitied the ingest speed. I can try that
again.
I also shutdown the tserver on one of those slow sync cost nodes and ingst
stopped for about 30 seconds and then continued at the same slow sp
When you shut down the two datanodes, did you have the same "slow sync
cost" messages concentrated on two nodes? If so, is it possible that a
majority of the writes are going to a small set of tablet servers? You
might be able to see this on the Monitor. Is it possible that tablets you
are ingestin
I stopped the two data nodes and it had no effect.
Thanks,
On Wed, Mar 15, 2023 at 6:53 PM Vincent Russell
wrote:
> Yes. We have the hdfs rack-aware set up to divide the blocks equally.
> And according to the name node http page it doesn't look like those nodes
> have a much higher number of b
Yes. We have the hdfs rack-aware set up to divide the blocks equally. And
according to the name node http page it doesn't look like those nodes have
a much higher number of blocks that nother nodes.
I can try temporarily shutting down one of the data nodes to see what that
does.
We did already
sounds like you have a hot-spot on those two datanode hosts. Either because
the blocks that it's writing to are all (or a majority) located there, or
there is some type of issue with the host. Stopping the DN processes on
those two hosts should confirm this, unless the hot spot moves. Do you have
t
Hello,
I am using accumulo 2.0.1 with hadoop 3.3.1.
I have two identical clusters with 28 tservers.
I have writers on both clusters which are set with 10 batch writers with a
max memory of 50m.
However, one server is ingesting 10x faster than the other.
Is there anything I should check for?
I
