Have you set -Xmx32g ? In this case you may get significantly less
available memory because of switch to 64-bit references. See
http://java-performance.info/over-32g-heap-java/ for details, and set
slightly less than 32Gb
Reid Pinchback at "Sun, 5 Apr 2020 00:50:43 +" wrote:
RP> Surbi:
RP
Hi Erick
Thank you very much for your friendly note.
ERROR [AntiEntropyStage:1] 2020-04-04 13:57:09,614
RepairMessageVerbHandler.java:177 - Table with id
21a3fa90-74c7-11ea-978a-b556b0c3a5ea was dropped during prepare phase of repair
cassandra@cqlsh:system_schema> select keyspace_name,table_name
Another suggestion before resetlocalschema. Try rolling restart all the
nodes in the cluster and see if it fix the problem. After the restart all
the nodes will use the same schema for the table.
On Sunday, April 5, 2020, David Ni wrote:
> Hi Erick
> Thank you very much for your friendly note.
>
>
> Another suggestion before resetlocalschema. Try rolling restart all the
> nodes in the cluster and see if it fix the problem. After the restart all
> the nodes will use the same schema for the table.
>
That's a little bit heavy-handed. 🙂 Resetting a node's schema is a simple,
online operation
Be really cautious here, this can be deceptive
There are races in some versions of cassandra that can leave you with different
combinations of cfid
The cfid is on disk for schema
It’s in memory for schema
It’s used for the table path on disk
Those three have to match for things to work properly
I just checked, we have setup the Heapsize to be 31GB not 32GB in DC2.
I checked the CPU and RAM both are same on all the nodes in DC1 and DC2.
What specific parameter I should check on OS ?
We are using CentOS release 6.10.
Currently disk_access_modeis not set hence it is auto in our env. Should
Set the jvm flags to heap dump on oom
Open up the result in a heap inspector of your preference (like yourkit or
similar)
Find a view that counts objects by total retained size. Take a screenshot. Send
that.
> On Apr 5, 2020, at 6:51 PM, Surbhi Gupta wrote:
>
> 
> I just checked, we hav
We are using JRE and not JDK , hence not able to take heap dump .
On Sun, 5 Apr 2020 at 19:21, Jeff Jirsa wrote:
>
> Set the jvm flags to heap dump on oom
>
> Open up the result in a heap inspector of your preference (like yourkit or
> similar)
>
> Find a view that counts objects by total retain
Nobody is going to do any better than guessing without a heap histogram
I’ve got pretty good intuition with cassandra in real prod environments and can
think of like 8-9 different possible causes, but none of them really stand out
as likely enough to describe in detail (mybe the memtable dea
