There is a a native kafka framework which runs on top of DC/OS. https://docs.mesosphere.com/service-docs/kafka/
This will most likely be a better way to run kafka on DC/OS rather than running it as a marathon framework. On Mon, Oct 2, 2017 at 7:35 AM, David Garcia <dav...@spiceworks.com> wrote: > I’m not sure how your requirements of Kafka are related to your > requirements for marathon. Kafka is a streaming-log system and marathon is > a scheduler. Mesos, as your resource manager, simply “manages” resources. > Are you asking about multitenancy? If so, I highly recommend that you > separate your Kafka cluster (and zookeeper) from your other services. > Kafka leverages the OS page cache to optimize read performance and it seems > likely this would interfere with Mesos resource management policy. > > -David > > On 10/2/17, 6:39 AM, "Valentin Forst" <valen...@aseno.de> wrote: > > Hi there, > > Working in a huge compony we are about to install Kafka on DC/OS > (Mesos) and intend to use Marathon as a Scheduler. Since I am new to DC/OS > and Marathon, I was wondering if this is a recommended way of using Kafka > in the production environment. > > My doubts are: > - Kafka manages Broker rebalancing (e.g. Failover, etc.) using its own > semantic. Can I trust Marathon that it will match the requirements here? > - Since our Container Platform - DC/OS is going to be used by other > „micro services“ - soon or later this is going to raise a performance > issue. Should we better use a dedicated DC/OS instance for our > Kafka-Cluster? Or Kafka-Cluster on its own? > - Is there something else we should consider important if using Kafka > on DC/OS + Marathon? > > > Thanks in advance for your time. > Valentin > > > >
