One of the problems I have experienced in the past has more to do with Java than Cassandra in particular, and that is the JVM ignoring cgroups. With Cassandra in particular I would often see memory usage go higher than what was desired. This would lead to pods getting oom killed. This was fixed in Java 10 though, and I believe even back ported to Java 8.
I think another issue is the lack of options for backup/restore. I worked with Cassandra in Kubernetes pre-StatefulSets. That was a bit rough :) Local volumes were promoted to GA in Kubernetes 1.14. That is certainly a good thing for stateful applications like Cassandra. It is also important to have a sufficiently large value for the termination grace period on pods to allow drain operation to complete, assuming you perform a drain on shutdown. On Wed, Oct 30, 2019 at 1:46 PM Akshit Jain <jainakshi...@gmail.com> wrote: > Hi Jean > Thanks for replying. I had seen CassKop and the amount functionality it > provides is quite awesome as compared to other operators. > > I would like to know how stable is kubernetes for stateful/database > applications right now? > > I haven't read/heard any major production stateful application running on > k8s. > > > -Akshit > > > > > On Wed, 30 Oct, 2019, 8:12 PM Jean-Armel Luce, <jaluc...@gmail.com> wrote: > >> Hi, >> >> We are currently developping CassKop, a Cassandra operator for K8S. >> This operator is developped in Go, based on the operator-sdk framework. >> >> At this time of the project, the goal is to deploy a Cassandra cluster in >> 1 Kubernetes datacenter, but this will change in next versions to deal with >> Kubernetes in multi-datacenters. >> >> The following features are already supported by CassKop: >> - Deployment of a C* cluster (rack or AZ aware) >> - Scaling up the cluster (with cleanup) >> - Scaling down the cluster (with decommission prior to Kubernetes scale >> down) >> - Pods operations (removenode, upgradesstable, cleanup, rebuild..) >> - Adding a Cassandra DC >> - Removing a Cassandra DC >> - Setting and modifying configuration files >> - Setting and modifying configuration parameters >> - Update of the Cassandra docker image >> - Rolling update of a Cassandra cluster >> - Update of Cassandra version (including upgradesstable in case of major >> upgrade) >> - Update of JVM >> - Update of configuration >> - Stopping a Kubernetes node for maintenance >> - Process a remove node (and create new Cassandra node on another >> Kubernetes node) >> - Process a replace address (of the old Cassandra node on another >> Kubernetes node) >> - Manage operations on pods through CassKop plugin (cleanup, rebuild, >> upgradesstable, removenode..) >> - Monitoring (using Instaclustr Prometheus exporter to Prometheus/Grafana) >> - Pause/Restart & rolling restart operations through CassKoP plugin. >> >> We use also Cassandra reaper for scheduling repair sessions. >> >> >> If you would like more informations about this operator, you may have a >> look here : https://github.com/Orange-OpenSource/cassandra-k8s-operator >> >> Please, feel free to download it and try it. We would be more than happy >> to receive your feedback >> >> >> If you have any question about this operator, feel free to contact us via >> our mailing-list: prj.casskop.supp...@list.orangeportails.net or on our >> slack https://casskop.slack.com >> >> Note : this operator is still in alpha version and works only in a mono >> region architecture for now. We are currently working hard for adding new >> features in order to run it in multi-regions architecture. >> >> >> Thanks. >> >> >> >> Le mer. 30 oct. 2019 à 13:56, Akshit Jain <jainakshi...@gmail.com> a >> écrit : >> >>> Hi everyone, >>> >>> Is there anyone who is running Cassandra on K8s clusters. It would be >>> great if you can share your experience , the operator you are using and the >>> overall stability of stateful sets in Kubernetes >>> >>> -Akshit >>> >> -- - John
