To dig a little deeper. It sounds like JMX is just enabled for loopback, if JMX was disabled/unreachable entirely, nodetool wouldn't work at all.
You will likely need to customize your Cassandra config to allow remote JMX connections (though this does increase your attack surface and risk from a security perspective). Alternatively, whatever monitoring solution you land on will likely need a local agent to connect to JMX via loopback. To answer your specific monitoring questions, iirc JMX will expose the number of active connections per node, however, it does not show what queries are being executed. Cassandra does support slow query logging (later versions), or you can use query tracing (probabilistic and at the session level) to see what is happening in real time. Though query tracing is fairly heavy from a performance perspective. I generally don't recommend thinking about tracking bad query issues in Cassandra the same way you would with a relational DB. Cassandra does not have long running queries that get stuck, hold resources, hold locks etc. Most bad performance smells from bad queries (as opposed to just an increase in load) will show up in the logs e.g. large batches, queries timing out, slow queries (when slow query logger is enabled), excessive/long GC pauses, etc However, you mention you get timeouts when you increase the load on Cassandra. If you had some monitoring in place to graph reads and writes per second against timeouts you might discover you simply need to add more nodes or look at trying to squeeze more performance out of what you have. Refer to Anups previous thoughts on where to get started with getting operational data out of Cassandra On Wed, Oct 31, 2018 at 8:21 PM Anup Shirolkar < anup.shirol...@instaclustr.com> wrote: > Hi, > > Looks like you need a monitoring for Cassandra but without using JMX. > It is possible to use metric reporting libraries in Cassandra: > https://wiki.apache.org/cassandra/Metrics#Reporting > > I do not have specific experience with using Cassandra on DCOS but, > monitoring with libraries and tools should not be any different. > > There are various options available to establish good monitoring > (Graphite, Prometheus, Grafana) > > Helpful links: > > > https://blog.pythian.com/monitoring-apache-cassandra-metrics-graphite-grafana/ > https://github.com/instaclustr/cassandra-exporter > https://prometheus.io/docs/instrumenting/exporters/ > https://grafana.com/dashboards/5408 > > http://thelastpickle.com/blog/2017/04/05/cassandra-graphite-measurements-filtered.html > > Regards, > > Anup Shirolkar > > > > > On Wed, 31 Oct 2018 at 18:41, Caesar, Maik <maik.cae...@dxc.com> wrote: > >> Hello All, >> >> have someone experience with monitoring cassandra in DCOS? >> >> If we increase the load to the Casandra in DCOS, the application get >> timeouts and loose the connection and I do not have any information about >> what happened. >> >> Is there a way to get information about the amount of current connection >> and which queries are executed? Cassandra in DCOS has disabled the JMX >> interface and I think the noodetool do not provide such information. >> >> >> >> Regards >> >> Maik >> >> >> >> DXC Technology Company -- This message is transmitted to you by or on >> behalf of DXC Technology Company or one of its affiliates. It is intended >> exclusively for the addressee. The substance of this message, along with >> any attachments, may contain proprietary, confidential or privileged >> information or information that is otherwise legally exempt from >> disclosure. Any unauthorized review, use, disclosure or distribution is >> prohibited. If you are not the intended recipient of this message, you are >> not authorized to read, print, retain, copy or disseminate any part of this >> message. If you have received this message in error, please destroy and >> delete all copies and notify the sender by return e-mail. Regardless of >> content, this e-mail shall not operate to bind DXC Technology Company or >> any of its affiliates to any order or other contract unless pursuant to >> explicit written agreement or government initiative expressly permitting >> the use of e-mail for such purpose. --. >> > -- Ben Bromhead CTO | Instaclustr <https://www.instaclustr.com/> +1 650 284 9692 Reliability at Scale Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
