Re: Why authorization deny Read ACL doesn't work

2017-06-12 Thread Tom Bentley
Hi, Did you set authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer as described here at http://docs.confluent.io/current/kafka/authorization.html#further-configuration HTH, Tom On 11 June 2017 at 04:40, linbo liao wrote: > Hi, > > I try to set Kafka ACL for topic access permissi

Re: Why authorization deny Read ACL doesn't work

2017-06-12 Thread linbo liao
Thanks Tom, I miss it. I added authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer, and grant user:alice to operation permission to cluster, but still failed when publish message. The error message told LEADER_NOT_AVAILABLE. $ bin/kafka-acls.sh --authorizer-properties > zookeeper.connec

Login using LAP for Kafka

2017-06-12 Thread Arunkumar
Hi I have a requirement to configure LDAP Login for Kafka. I see there is a discussion going on to integrate it with future release of Kafka version. Is there any one who have done it ( I am sure some one would have done it ):- ) can help with sample configuration with code. I see we can exten

Re: Why authorization deny Read ACL doesn't work

2017-06-12 Thread linbo liao
Finally I figure it out, I miss add super user in config/server.properties. Now everything works fine. 2017-06-12 19:19 GMT+08:00 linbo liao : > Thanks Tom, I miss it. > > I added authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer, > and grant user:alice to operation permission to clus

Re: Kafka connector throughput reduction upon avro schema change

2017-06-12 Thread Dave Hamilton
Hi, does anyone have advice on how to deal with this issue? Is it possible that changing a schema compatibility setting could correct it? Thanks, Dave On 5/26/17, 1:44 PM, "Dave Hamilton" wrote: We are currently using the Kafka S3 connector to ship Avro data to S3. We made a change to on

kafka streams first time did not print output

2017-06-12 Thread john cheng
Hi there, I'm testing Kafka Streams's print() method, here is the code: props.put(StreamsConfig.APPLICATION_ID_CONFIG, "dsl-wc1"); props.put(StreamsConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG, 0); KStream source = builder.stream("dsl-input1"); KTable countTable = source

Re: [DISCUSS] KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

2017-06-12 Thread Colin McCabe
Hi Vahid, I think you make a valid point that the ACLs controlling group operations are not very intuitive. This is probably a dumb question, but why are we using Read for mutating APIs? Shouldn't that be Write? The distinction between Describe and Read makes a lot of sense for Topics. A group

Re: [DISCUSS] KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

2017-06-12 Thread Vahid S Hashemian
Hi Colin, Thanks for the feedback. To be honest, I'm not sure either why Read was selected instead of Write for mutating APIs in the initial design (I asked Ewen on the corresponding JIRA and he seemed unsure too). Perhaps someone who was involved in the design can clarify. Thanks. --Vahid

Re: Kafka 0.10.1 cluster using 100% disk usage (reads)

2017-06-12 Thread Milind Vaidya
This sound exactly similar to what I experienced in the similar scenario. Can you please take a look at the File System time stamp of the actual log files on one of the broker hosts ? For me when I restarted the new brokers with version 0.10.0 it changes to current ts. Meaning if I have set 48 hr

mirror-maker-thread-failure

2017-06-12 Thread Fernando Vega
Does anyone has experience or had similar issue like https://community.hortonworks.com/questions/106969/mirror-maker-thread-failure-due-to-kafkatoolsmirro.html If you can point me to something that I can try that will be highly appreciated Thank you

RE: Kafka 0.10.1 cluster using 100% disk usage (reads)

2017-06-12 Thread ext-gfenol...@eramet-sln.nc
Hello Milind, and thank you for your answer. We just found the solution to our problem. Apparently, some developers were sending bad messages with no key, so the log cleaner couldn’t wipe them properly and crashed at startup. The topics were set in compact mode. We have set the compact.policy to

Slow Consumer Group Startup

2017-06-12 Thread Bryan Baugher
Hi everyone, We are currently experiencing slow startup times for our consumer groups (16-32 processes for a hundred or more partitions) in the range of minutes (3-15 minutes), where little to no messages are consumed before suddenly everything just starts working at full speed. I'm currently usi

Re: Kafka 0.10.1 cluster using 100% disk usage (reads)

2017-06-12 Thread Ismael Juma
For what is worth, 0.9.0.0 and above don't accept messages with no key for compacted topics. Ismael On Mon, Jun 12, 2017 at 10:17 PM, ext-gfenol...@eramet-sln.nc < ext-gfenol...@eramet-sln.nc> wrote: > Hello Milind, and thank you for your answer. > > We just found the solution to our problem. >

Re: Slow Consumer Group Startup

2017-06-12 Thread J Pai
When are the topics on which these consumer groups consume, created? -Jaikiran On 13-Jun-2017, at 3:18 AM, Bryan Baugher wrote: Hi everyone, We are currently experiencing slow startup times for our consumer groups (16-32 processes for a hundred or more partitions) in the range of minutes (3-15