Providing an official docker image is a little tricky, as despite what container marketing would tell you, containers need to make assumptions about outside orchestration/management methods. Folks in this thread have already identified differences in kubernetes distro's let alone other container schedulers.
For example do you provide configuration via environment variables, docker configs, kubernetes configmap/secret mechanism etc? Does your container environment allow you to run containers as root or specify kernel capabilities? C* has some explicit requirements around what it expects of the OS in that regard for stability and performance. What base image and userland tools do you use? On top of that, many operators (folks that run C*, not the k8s concept) need to embed their own sidecars and agents. For example embedding the jolokia agent vs using JMX vs using a prometheus exporter etc. A quick survey across the ecosystem shows that there are wildly different opinions about what a cassandra container should look like and include: - https://github.com/docker-library/cassandra - https://github.com/datastax/docker-images - https://github.com/thelastpickle/docker-cassandra-bootstrap - https://github.com/bitnami/bitnami-docker-cassandra - https://github.com/instaclustr/cassandra-operator/tree/master/docker/cassandra That's not to say it can't be done, it's just it creates an explicit and rigid contract in the way containers are configured, what's included and how things are managed. Don't take this commentary as me being against the idea, just based on my experience, it will be hard to meet even a slim majority of users production requirements for an image :) On Thu, Sep 19, 2019 at 5:38 PM Dimo Velev <dimo.ve...@gmail.com> wrote: > Sidecar would be fine I guess provided that it does not require a lot of > resources - which kind of eliminates more or less any java implementation. > Unsure whether jvm metrics etc are available via virtual tables though. > > Also forgot an important point about the image for local usage - ability > to load cql scripts upon start only once. Most databases offer these so > that you can add initialization stuff like creating keyspaces, users, > tables upon first start. > > > On 19. Sep 2019, at 23:02, Jon Haddad <j...@jonhaddad.com> wrote: > > > > I recently had a side conversation about including a Prometheus endpoint > in > > the sidecar project, which would query virtual tables exclusively. Is it > > reasonable to require running the sidecar in addition to Cassandra when > > deploying on K8? My container knowledge is pretty limited. > > > > If that's not a problem, I think it shouldn't be too hard to add, it's > just > > a matter of someone finding the time. > > > > On Thu, Sep 19, 2019 at 1:39 PM Dimo Velev <dimo.ve...@gmail.com> wrote: > > > >> Hi, > >> > >> A docker image for Cassandra would be the first step in the right > >> direction. What one would really want is a kubernetes Operator that can > >> deploy, upgrade, etc. a Cassandra cluster ( > >> https://kubernetes.io/docs/concepts/extend-kubernetes/operator). > >> > >> A base image with placeholders would be neat for people to quickly start > >> using Cassandra. We have built our own image (actually a few of them > over > >> the time) and also use it e.g. for cqlsh (docker run -it —rm —entrypoint > >> cqlsh <image> —help). What one would probably do in the end is building > >> their own image based on the official one adding a layer with the > >> configuration file, certificates, etc (or mounting them from a > configmap in > >> kubernetes). > >> > >> Some things to consider for the image: > >> • Special care must be taken to make the image usable in OpenShift as it > >> does not run the processes in the containers as root. > >> • Logging should be changed to stdout so that it is easier to use and > >> automatically picked up by log indexers of kubernetes clusters. > >> • As for metrics, exposing Prometheus endpoint with them would be great > as > >> this makes scraping them just a matter of meta configuration of the pods > >> (telling Prometheus the actual endpoint to scrape) > >> • I had issues with Cassandra binding on the wrong IP in a container > >> > >> Would be happy to help with the image part / testing on OpenShift. > >> > >> Cheers, > >> Dimo > >> > >>> On 19. Sep 2019, at 18:32, Nate McCall <zznat...@gmail.com> wrote: > >>> > >>> Hi Cyril, > >>> Thanks for bringing this topic up. I think it would be a good idea for > us > >>> to have an "official" docker file in the source tree. > >>> > >>> There are, however, some caveats: > >>> > >> > https://issues.apache.org/jira/browse/LEGAL-270?focusedCommentId=15524446&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-15524446 > >>> > >>> As long as we could adhere to those guidelines (which I don't see as > >> hard) > >>> we could do this. > >>> > >>> In looking through your specific image (thanks for posting this, btw), > I > >>> would personally prefer something with a lot fewer dependencies > >> (basically > >>> from just our source tree) and a lot more replacement properties > >> available > >>> for config files. > >>> > >>> Curious what other folks think? > >>> > >>> Cheers, > >>> -Nate > >>> > >>>> On Wed, Sep 18, 2019 at 6:43 AM Cyril Scetbon <cyril.scet...@free.fr> > >> wrote: > >>>> > >>>> Hey guys, > >>>> > >>>> I heard that at the last summit there were discussions about providing > >> an > >>>> official docker image to run Cassandra on Kubernetes. Is it something > >> that > >>>> you’ve started to work on ? We have our own at > >>>> https://github.com/Orange-OpenSource/cassandra-image < > >>>> https://github.com/Orange-OpenSource/cassandra-image> but I think > >>>> providing an official image makes sense. As long as we can easily do > >>>> everything we do today. We could also collaborate. > >>>> > >>>> Thank you > >>>> — > >>>> Cyril Scetbon > >>>> > >>>> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > -- Ben Bromhead Instaclustr | www.instaclustr.com | @instaclustr <http://twitter.com/instaclustr> | (650) 284 9692
