Hi Hangjun
I've explored deploying kafka on yarn and current YARN does not support long
running services with locality constraints. Deploying kafka producers /
consumers (not brokers) is supported in the apache incubator samza project.
Background on YARN limitations can be found here: YARN-371,
[
https://issues.apache.org/jira/browse/KAFKA-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748147#comment-13748147
]
Kam Kasravi commented on KAFKA-949:
---
This work is available under https://github
Thanks - I'll ask on bigtop regarding the .deb requirement - it seems they
don't abide by this.
I may merge a bit of your work into bigtop-989 if that's ok with you. I do know
the bigtop folks
would like to see sbt support.
From: Andrew Otto
Thanks Andrew - I like the shell wrapper - very clean and simple.
What installs all the kafka dependencies under /usr/share/java?
From: Andrew Otto
To: Kam Kasravi
Cc: "dev@kafka.apache.org" ; Ken Goodhope
; Andrew Psaltis ;
"dib
Thanks Andrew. I upgrade it to use the high level consumer.
From: Andrew Psaltis
To: "us...@kafka.apache.org" ; Kam Kasravi
; "dev@kafka.apache.org" ; Ken
Goodhope
Cc: Andrew Psaltis ;
"dibyendu.bhattacha...@pearson.com"
I would like to do this refactoring since I did a high level consumer a while
ago.
A few weeks ago I had opened KAFKA-949 Kafka on Yarn which I was also hoping to
add to contribute.
It's almost done. KAFKA-949 is paired with BIGTOP-989 which adds kafka 0.8 to
the bigtop distribution.
KAFKA-949
Kam Kasravi created KAFKA-949:
-
Summary: Integrate kafka into YARN
Key: KAFKA-949
URL: https://issues.apache.org/jira/browse/KAFKA-949
Project: Kafka
Issue Type: New Feature
Components
[
https://issues.apache.org/jira/browse/KAFKA-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kam Kasravi resolved KAFKA-884.
---
Resolution: Fixed
That resolved it. Thanks Neha.
> Get java.lang.NoSuchMethodEr
Kam Kasravi created KAFKA-884:
-
Summary: Get java.lang.NoSuchMethodError:
com.yammer.metrics.core.TimerContext.stop()J when stopping kafka brokers
Key: KAFKA-884
URL: https://issues.apache.org/jira/browse/KAFKA-884
