The official Kubernetes Java driver is actually pretty feature complete, if not exactly idiomatic Java... it's only missing full examples to get it to GOLD compatibility levels iirc.
A few reasons we went down the Java path: - Cassandra community engagement was the primary concern. If you are a developer in the Cassandra community you have a base level of Java knowledge, so it means if you want to work on the Kubernetes operator you only have to learn 1 thing, Kubernetes. If the operator was in Go, you would then have two things to learn, Go and Kubernetes :) - We actually wrote an initial PoC in Go (based off the etcd operator, you can find it here https://github.com/benbromhead/cassandra-operator-old ), but because it was in Go we ended up making architectural decisions simply because Go doesn't do JMX, so it felt like we were just fighting different ecosystems just to be part of the cool group. Some other less important points weighed the decision in Java's favour: - The folk at Instaclustr all know Java, and are productive in it from day 1. Go is fun and relatively simple, but not our forte. - <troll> Mature package management, Generics/inability to write DRY code, a million if err statements </troll> (: - Some other awesome operators/controllers are written in JVM based languages. The sparkKubernetes resource manager (which is a k8s controller) is written in Scala. On Wed, May 23, 2018 at 10:04 AM vincent gromakowski < vincent.gromakow...@gmail.com> wrote: > Why did you choose java for the operator implementation when everybody > seems to use the go client (probably for greater functionalities) ? > > 2018-05-23 15:39 GMT+02:00 Ben Bromhead <b...@instaclustr.com>: > >> You can get a good way with StatefulSets, but as Tom mentioned there are >> still some issues with this, particularly around scaling up and down. >> >> We are working on an Operator for Apache Cassandra, you can find it here >> https://github.com/instaclustr/cassandra-operator. This is a joint >> project between Instaclustr, Pivotal and a few other folk. >> >> Currently it's a work in progress, but we would love any or all early >> feedback/PRs/issues etc. Our first GA release will target the following >> capabilities: >> >> - Safe scaling up and down (including decommissioning) >> - Backup/restore workflow (snapshots only initially) >> - Built in prometheus integration and discovery >> >> Other features like repair, better PV support, maybe even a nice >> dashboard will be on the way. >> >> >> On Wed, May 23, 2018 at 7:35 AM Tom Petracca <tpetra...@palantir.com> >> wrote: >> >>> Using a statefulset should get you pretty far, though will likely be >>> less effective than a coreos-style “operator”. Some random points: >>> >>> - For scale-up: a node shouldn’t report “ready” until it’s in the >>> NORMAL state; this will prevent multiple nodes from bootstrapping at >>> once. >>> - For scale-down: as of now there isn’t a mechanism to know if a pod >>> is getting decommissioned because you’ve permanently lowered replica >>> count, >>> or because it’s just getting bounced/re-scheduled, thus knowing whether >>> or >>> not to decommission is basically impossible. Relevant issue: >>> kubernetes/kubernetes#1462 >>> <https://github.com/kubernetes/kubernetes/issues/1462> >>> >>> >>> >>> *From: *Pradeep Chhetri <prad...@stashaway.com> >>> *Reply-To: *"user@cassandra.apache.org" <user@cassandra.apache.org> >>> *Date: *Friday, May 18, 2018 at 10:20 AM >>> *To: *"user@cassandra.apache.org" <user@cassandra.apache.org> >>> *Subject: *Re: Using K8s to Manage Cassandra in Production >>> >>> >>> >>> Hello Hassaan, >>> >>> >>> >>> We use cassandra helm chart[0] for deploying cassandra over kubernetes >>> in production. We have around 200GB cas data. It works really well. You can >>> scale up nodes easily (I haven't tested scaling down). >>> >>> >>> >>> I would say that if you are worried about running cassandra over k8s in >>> production, maybe you should first try setting it for your >>> staging/preproduction and gain confidence over time. >>> >>> >>> >>> I have tested situations where i have killed the host running cassandra >>> container and have seen that container moves to a different node and joins >>> cluster properly. So from my experience its pretty good. No issues till yet. >>> >>> >>> >>> [0]: https://github.com/kubernetes/charts/tree/master/incubator/cassandra >>> [github.com] >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_charts_tree_master_incubator_cassandra&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=1oh1YI8i5eJD1DFTwooO7U92fFi2fjan6lqP61yAiyo&m=dupKDpZi0lkjFkqaSd6XaEj5nuY1T5UObgAcXCBqo7A&s=0WTYStEM1zvh2BQKvnVLRpukxgr0aDLyGffyE1V2xik&e=> >>> >>> >>> >>> >>> >>> Regards, >>> >>> Pradeep >>> >>> >>> >>> On Fri, May 18, 2018 at 1:01 PM, Павел Сапежко <amelius0...@gmail.com> >>> wrote: >>> >>> Hi, Hassaan! For example we are using C* in k8s in production for our >>> video surveillance system. Moreover, we are using Ceph RBD as our storage >>> for cassandra. Today we have 8 C* nodes each manages 2Tb of data. >>> >>> >>> >>> On Fri, May 18, 2018 at 9:27 AM Hassaan Pasha <hpa...@an10.io> wrote: >>> >>> Hi, >>> >>> >>> >>> I am trying to craft a deployment strategy for deploying and maintaining >>> a C* cluster. I was wondering if there are actual production deployments of >>> C* using K8s as the orchestration layer. >>> >>> >>> >>> I have been given the impression that K8s managing a C* cluster can be a >>> recipe for disaster, especially if you aren't well versed with the >>> intricacies of a scale-up/down event. I know use cases where people are >>> using Mesos or a custom tool built with terraform/chef etc to run their >>> production clusters but have yet to find a real K8s use case. >>> >>> >>> >>> *Questions?* >>> >>> Is K8s a reasonable choice for managing a production C* cluster? >>> >>> Are there documented use cases for this? >>> >>> >>> >>> Any help would be greatly appreciated. >>> >>> >>> >>> -- >>> >>> Regards, >>> >>> >>> >>> *Hassaan Pasha* >>> >>> -- >>> >>> Regrads, >>> >>> Pavel Sapezhko >>> >>> >>> >> -- >> Ben Bromhead >> CTO | Instaclustr <https://www.instaclustr.com/> >> +1 650 284 9692 <(650)%20284-9692> >> Reliability at Scale >> Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer >> > > -- Ben Bromhead CTO | Instaclustr <https://www.instaclustr.com/> +1 650 284 9692 Reliability at Scale Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
