Re: Is there any command can be used to get broker.id on one of broker with command?

2017-02-16 Thread Manikumar
If broker id is not set, then a unique broker id will be generated. This is stored in meta.properties file of data logs directory. This is id is also available through JMX mbean. On Thu, Feb 16, 2017 at 12:32 PM, Stevo Slavić wrote: > You can just grep it from broker's config/server.properties >

Re: [kafka-clients] [VOTE] 0.10.2.0 RC2

2017-02-16 Thread Rajini Sivaram
+1 (non-binding) Ran quick start and some security tests on binary, checked source build and tests. Thank you, Rajini On Thu, Feb 16, 2017 at 2:04 AM, Jun Rao wrote: > Hi, Ewen, > > Thanks for running the release. +1. Verified quickstart on 2.10 binary. > > Jun > > On Tue, Feb 14, 2017 at 10:

Re: [kafka-clients] [VOTE] 0.10.2.0 RC2

2017-02-16 Thread Tom Crayford
+1 (non-binding) I didn't explicitly state my voting status in my previous comment, sorry. On Thu, Feb 16, 2017 at 1:59 PM, Rajini Sivaram wrote: > +1 (non-binding) > > Ran quick start and some security tests on binary, checked source build and > tests. > > Thank you, > > Rajini > > On Thu, Feb

Re: [kafka-clients] [VOTE] 0.10.2.0 RC2

2017-02-16 Thread Manikumar
+1 (non-binding) Verified quick start, artifacts On Thu, Feb 16, 2017 at 7:33 PM, Tom Crayford wrote: > +1 (non-binding) > > I didn't explicitly state my voting status in my previous comment, sorry. > > On Thu, Feb 16, 2017 at 1:59 PM, Rajini Sivaram > wrote: > > > +1 (non-binding) > > > > Ran

Re: Changing offsets.retention.minutes

2017-02-16 Thread Neelesh
The command succeeds, but does not have an impact. Setting it to a minute does not clear the logs for this topic. The code in GroupMetadataManager also does not seem to support it. On Feb 15, 2017 10:31 PM, "Manikumar" wrote: > Yes, we can change. > > On Thu, Feb 16, 2017 at 7:09 AM, Neelesh wr

Re: Changing offsets.retention.minutes

2017-02-16 Thread Manikumar
Yes, Sorry. I was answering for "log.retention.minutes" config. I think, we also need to update the doc for "offsets.retention.minutes" config. It is not "Log retention window for offsets topic". It is for discarding offsets older than retention period. On Thu, Feb 16, 2017 at 9:56 PM, Neelesh wr

Re: Changing offsets.retention.minutes

2017-02-16 Thread Neelesh
Thanks. Looking at the code, it looks like for offsetandmeta version > 0, the offset.retention.minutes config is not retroactive either, because the expireTimestamp is set during commit . For older offsetandmeta requests which do not have a commit timestamp, it's computed while loading offsets, hen

Re: Is there any command can be used to get broker.id on one of broker with command?

2017-02-16 Thread Andrew Otto
Best I got for ya, using https://github.com/r4um/jmx-dump $ broker_hostname=localhost $ jmx_port= $ java -jar jmx-dump-0.4.2-standalone.jar -h $broker_hostname -p $jmx_port -m | grep ‘kafka.server:id=.*type=app-info’ kafka.server:id=12,type=app-info This will only work if a JMX port is config

Role of Zookeeper in Kafka Rest Proxy

2017-02-16 Thread Stephane Maarek
Hi, What is Zookeeper used for in the Kafka Rest Proxy? Is the dependency on Zookeeper why I can’t integrate Kafka Rest proxy with a secure cluster? http://docs.confluent.io/3.1.1/kafka-rest/docs/security.html Regards, Stephane

Re: Kerberos/SASL Enabled Kafka - broker fails NoAuth due ACL

2017-02-16 Thread Stephane Maarek
So the issue is that you need to have your kafka/f...@realm.com in the KafkaServer jaas part, but the same zkcli...@realm.com in the Client jaas part. That should solve your issues On 9 February 2017 at 7:42:54 pm, Ashish Bhushan (ashish6...@gmail.com) wrote: Any help ? On 09-Feb-2017 1:13 PM,

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

2017-02-16 Thread Vahid S Hashemian
Thanks for this useful and the nice KIP write-up. Just a minor suggestion. Would it make sense to avoid repeating the term "reset" in the arguments? We already use the argument "reset-offset", so we may not need to repeat the term in the follow-on argument. For example, instead of kafka-consum

Can't produce message with SASL: timeout error

2017-02-16 Thread Stephane Maarek
Hi, I secured my cluster and everything was working fine. Brokers are up and don’t complain, my topics are all synchronized. Here’s my config (excerpt): listeners= PLAINTEXT://0.0.0.0:9092,SSL://0.0.0.0:9093,SASL_PLAINTEXT://0.0.0.0:9094,SASL_SSL://0.0.0.0:9095 super.users=User:kafka;User:ANONYMO

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

2017-02-16 Thread James Cheng
Yeah, that's a good point. Some of the operations might make sense on multiple partitions at once. Moving to a timestamp might apply to all partitions, moving backwards and forwards by N offsets might apply to all partitions. However, moving to a specific offset ("set to offset 43") would most

Re: Can't produce message with SASL: timeout error

2017-02-16 Thread Manikumar
Pl enable authorizer logs ( config/log4j.properties) and check if operations are getting denied. Also, enable producer debug logs (config/tools-log4j.properties) to check for any errors. On Fri, Feb 17, 2017 at 10:34 AM, Stephane Maarek < steph...@simplemachines.com.au> wrote: > Hi, > > I secure

Re: Can't produce message with SASL: timeout error

2017-02-16 Thread Stephane Maarek
It seems to be conflicting when I try running the commands on the same box as where Kafka is running. I think it was inheriting Kafka's principal somehow and not the kafkaclient principal. When moving to another box all seems to work ! An odd one I see if I can reproduce and open a JIRA On 17 Feb.

Re: Role of Zookeeper in Kafka Rest Proxy

2017-02-16 Thread hans
You can't integrate 3.1.1 REST Proxy with a secure cluster because it uses the old consumer API (hence zookeeper dependency). The 3.2 REST Proxy will allow you to integrate with a secure cluster because it is updated with the latest 0.10.2 client. -hans Sent from my iPhone > On Feb 16, 2017,