Re: Kafka 11 | Stream Application crashed the brokers

2017-09-08 Thread Sameer Kumar
Hi All, Any thoughts on the below mail. -Sameer. On Wed, Sep 6, 2017 at 12:28 PM, Sameer Kumar wrote: > Hi All, > > I want to report a scenario wherein my running 2 different instances of my > stream application caused my brokers to crash and eventually my stream > application as well. This s

Re: Reduce Kafka Client logging

2017-09-08 Thread Kamal Chandraprakash
add this lines at the end of your log4j.properties, log4j.logger.org.apache.kafka.clients.producer=WARN On Thu, Sep 7, 2017 at 5:27 PM, Raghav wrote: > Hi Viktor > > Can you pleas share the log4j config snippet that I should use. My Java > code's current log4j looks like this. How should I add

KTable-KTable Join Semantics on NULL Key

2017-09-08 Thread Kamal Chandraprakash
Hi Kafka Users, KTable-KTable Join Semantics is explained in detailed [here][1]. But, it's not clear when the input record is , some times the output records are generated and in some cases it's not. It will be helpful, if someone explain on how the output records are generated for all the 3

Re: KTable-KTable Join Semantics on NULL Key

2017-09-08 Thread Damian Guy
It is shown in the table what happens when you get null values for a key. On Fri, 8 Sep 2017 at 12:31 Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi Kafka Users, > > KTable-KTable Join Semantics is explained in detailed [here][1]. But, > it's not clear when the input recor

kafka command line browser

2017-09-08 Thread Craig Swank
Hello, I've been working on a command line tool for browsing kafka that I find useful. I'd just like to share here it in case others find it useful too. The source is at https://github.com/cswank/kcli and you can see a video of it in action at: https://asciinema.org/a/7wobtflusydvswbsn1d320o1g.

when will zk path "/kafka/brokers/topics/__consumer_offsets/partitions/{number}/state" be updated

2017-09-08 Thread Nick
Dears, May I seek your help about below issue? Currently, I have met a issue as below: 1) I start a zk from docker as below: 2) And then I start the kafka from docker as below: 3) Every work fine, msg can be produced/consumed to/from kafka successfully 4) Then I remove the kafka w

Upgrading Kafka to 11.0

2017-09-08 Thread kiran kumar
Hi All, 1. Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. What does it mean to "update the code". Does it mean replace the old lib folder with latest ? or replace lib and bin with latest? Could someone clarify ? Thanks, Kiran

Re: Reduce Kafka Client logging

2017-09-08 Thread Raghav
Thanks, Kamal. On Fri, Sep 8, 2017 at 4:10 AM, Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > add this lines at the end of your log4j.properties, > > log4j.logger.org.apache.kafka.clients.producer=WARN > > On Thu, Sep 7, 2017 at 5:27 PM, Raghav wrote: > > > Hi Viktor > > > > Can

Re: Re: Using ACLs without Kerberos

2017-09-08 Thread Manoj Murumkar
This is fixed as well, btw. My ZK config wasn't done properly. On Tue, Sep 5, 2017 at 11:08 PM, Manoj Murumkar wrote: > Thanks, SASL setup for ZK does the trick. > > Now, I am continuing the security setup with schema registry. I am > configuring it to use same PLAINTEXT method for now. > > list

Re: when will zk path "/kafka/brokers/topics/__consumer_offsets/partitions/{number}/state" be updated

2017-09-08 Thread Ted Yu
The logs didn't go through. Consider using pastebin. For steps 1 and 2, it seems you used some image which didn't go through. Please use pastebin instead. Where did you get the docker image ? Cheers On Fri, Sep 8, 2017 at 4:42 AM, Nick <394299...@qq.com> wrote: > Dears, > > May I seek your hel

Re: Kafka 11 | Stream Application crashed the brokers

2017-09-08 Thread Guozhang Wang
Hello Sameer, I looked through your code, and here is what I figured: in 0.11 version we added the exactly-once feature ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging ) Which uses the transaction log (internal topic named "__transact

Classloading Error with Kotlin and Streams

2017-09-08 Thread Avi Flax
Hi all, I’m trying to experiment with Kotlin and Streams, and I’m encountering an error: $ kotlinc -cp kafka-streams-0.11.0.0-cp1.jar:kafka-clients-0.11.0.0-cp1.jar:slf4j-api-1.7.25.jar -jvm-target 1.8 Welcome to Kotlin version 1.1.4-3 (JRE 1.8.0_141-b15) Type :help for help, :quit for quit >>

Re: Classloading Error with Kotlin and Streams

2017-09-08 Thread Bill Bejeck
Hi Avi, I'm not super familiar with Kotiln, but I'm taking a look. Thanks, Bill On Fri, Sep 8, 2017 at 4:28 PM, Avi Flax wrote: > Hi all, I’m trying to experiment with Kotlin and Streams, and I’m > encountering an error: > > $ kotlinc -cp > kafka-streams-0.11.0.0-cp1.jar:kafka-clients-0.11.0.

Re: WorkerSinkTask not committing offsets (potential bug)

2017-09-08 Thread Shrijeet Paliwal
I found that this was already reported here https://issues.apache.org/jira/browse/KAFKA-5731 and fixed! -- Shrijeet On Thu, Sep 7, 2017 at 6:04 PM, Shrijeet Paliwal wrote: > > *Kafka Version: 0.10.2.1* > > Hi, > > I am running a custom connector (in distributed mode) and noticed one of > the pa

Re: [VOTE] 0.11.0.1 RC0

2017-09-08 Thread Guozhang Wang
Verified the quickstart and streams tutorial, +1. One minor comment on the web docs of streams tutorial, that when I edited the mvn archetype command I used the template values as: ``` mvn archetype:generate \ -DarchetypeGroupId=org.apache.kafka \ -DarchetypeArtifactId=strea