[
https://issues.apache.org/jira/browse/KAFKA-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197448#comment-14197448
]
Thomas Weise commented on KAFKA-1754:
-------------------------------------
YARN-896 references many of the challenges for long running applications and
services and is also indicative that we are on the way to overcome those. For
most part, there are ways to deal with those today until direct support becomes
available in YARN itself. We at DataTorrent have been working on a YARN native
platform for long running applications for ~2.5 years, in fact we aim to
support applications that never go down! For long running services, initiatives
such as Llama and Slider show how long running applications and services can be
supported on YARN as available today. We have a working prototype for KOYA and
looking forward to continue the effort in open source. We are looking forward
to collaborate with everyone interested to run Kafka on YARN!
> KOYA - Kafka on YARN
> --------------------
>
> Key: KAFKA-1754
> URL: https://issues.apache.org/jira/browse/KAFKA-1754
> Project: Kafka
> Issue Type: New Feature
> Reporter: Thomas Weise
> Attachments: DT-KOYA-Proposal- JIRA.pdf
>
>
> YARN (Hadoop 2.x) has enabled clusters to be used for a variety of workloads,
> emerging as distributed operating system for big data applications.
> Initiatives are on the way to bring long running services under the YARN
> umbrella, leveraging it for centralized resource management and operations
> ([YARN-896] and examples such as HBase, Accumulo or Memcached through
> Slider). This JIRA is to propose KOYA (Kafka On Yarn), a YARN application
> master to launch and manage Kafka clusters running on YARN. Brokers will use
> resources allocated through YARN with support for recovery, monitoring etc.
> Please see attached for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
