There is a a native kafka framework which runs on top of DC/OS.

https://docs.mesosphere.com/service-docs/kafka/

This will most likely be a better way to run kafka on DC/OS rather than
running it as a marathon framework.



On Mon, Oct 2, 2017 at 7:35 AM, David Garcia <dav...@spiceworks.com> wrote:

> I’m not sure how your requirements of Kafka are related to your
> requirements for marathon.  Kafka is a streaming-log system and marathon is
> a scheduler.  Mesos, as your resource manager, simply “manages” resources.
> Are you asking about multitenancy?  If so, I highly recommend that you
> separate your Kafka cluster (and zookeeper) from your other services.
> Kafka leverages the OS page cache to optimize read performance and it seems
> likely this would interfere with Mesos resource management policy.
>
> -David
>
> On 10/2/17, 6:39 AM, "Valentin Forst" <valen...@aseno.de> wrote:
>
>     Hi there,
>
>     Working in a huge compony we are about to install Kafka on DC/OS
> (Mesos) and intend to use Marathon as a Scheduler. Since I am new to DC/OS
> and Marathon, I was wondering if this is a recommended way of using Kafka
> in the production environment.
>
>     My doubts are:
>     - Kafka manages Broker rebalancing (e.g. Failover, etc.) using its own
> semantic. Can I trust Marathon that it will match the requirements here?
>     - Since our Container Platform - DC/OS is going to be used by other
> „micro services“ - soon or later this is going to raise a performance
> issue. Should we better use a dedicated DC/OS instance for our
> Kafka-Cluster? Or Kafka-Cluster on its own?
>     - Is there something else we should consider important if using Kafka
> on DC/OS + Marathon?
>
>
>     Thanks in advance for your time.
>     Valentin
>
>
>
>

Reply via email to