On Wed, Jul 13, 2016 at 10:08 PM, Jon Brisbin <jbris...@basho.com> wrote: > I guess to answer at least one part of your question on whether it's > "encouraged" or not: it depends on what you want to do with it. > > I can't speak for the support side of things but on the engineering side > anyway we're using Docker containers fairly happily to do things like create > ephemeral clusters for doing adhoc testing. It makes for a very nice way to > run a small cluster on a laptop or on a Mesos slave in a Jenkins build with > very similar expectations (I won't say identical because networking issues > can rear their head). I created images based on the latest TS and KV > packages. There are docs in the READMEs: > > https://hub.docker.com/r/basho/riak-ts/ > https://hub.docker.com/r/basho/riak-kv/ > > Running a production Riak cluster, however, raises some concerns that I'm > pretty sure no one really has a good handle on. Networking issues can become > difficult unless you use something like Weave. Volumes could also be an > issue. docker-compose and Docker Swarm should help with some of this but we > don't have any docs that demonstrate how to use them.
I'd say there's two classes of issues here: Most of the networking issues arise from the use of distributed Erlang: the Erlang port mapper deamon requires that you operate in host networking mode, on a known port visible to all other nodes, because of the way node discovery and clustering works in Erlang. Outside of that, if you're running multiple containers on the same machine, you'll also need to modify Riak to listen on different ports for the Protocol Buffer and HTTP interfaces, because under host networking only one server is going to be able to bind. The second is the lifecycle around the containers: where is the persistent storage going to be for the cluster? How will cluster orchestrations in Riak be handled? Mesos, in it's current iteration does not have a mechanism for issuing a callback when a container terminates, so an explicit operator command is still required to force remove a node from the cluster if a container is to die. How will nodes be added to the cluster when a new container is launched? Will an orchestration framework such as Mesos or Kubernetes be used? The operational side is very unclear at this point. > I'd say we also don't have a comprehensive picture of performance > characteristics of running a Dockerized Riak cluster since there's several > moving parts to consider. We need to investigate bridged vs host networking > mode and using different volume drivers, etc... There is some valid concern > that the key reasons you'd want to use Riak to begin with are somewhat > ameliorated by a Docker container's ephemeral nature; e.g. it's really easy > to kill off containers which might lead to accidental data loss. As someone who's been doing a significant amount of Erlang in Docker, I'd also be worried about the oom-killer and cgroups with Docker containers if one is considering Mesos: you don't want to lose a node and risk data loss if your node happens to have a memory fluctuation. I've personally made some headway in this for my own research when migrating some of our application off of Riak Core and it's involved finding solutions for many of these things: it's lead us down the path of completely disabling distributed Erlang and doing explicit TCP connection handling between nodes; using a custom membership protocol; and writing a system for container lifecycle management tied into membership information being exported from Kubernetes/Mesos (and, I'm sure similar could be done for swarm, weave, etc.) Thanks, - Christopher _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
