Re: Kafka behavior when consuming a topic which doesn't exist?

2017-07-07 Thread Ali Akhtar
What happens if auto creation is enabled but the topic doesn't exist. Consumers subscribe to that topic which doesn't exist. Then messages are posted to that topic. Will the consumer receive those messages in this scenario? On 8 Jul 2017 4:38 a.m., "M. Manna" wrote: That depends. If auto cre

Re: Hello, Help!

2017-07-07 Thread jan
Hi, I'd is this the right place to ask about cockroachDB? (well he started it, officer...) jan On 07/07/2017, David Garcia wrote: > “…events so timely that the bearing upon of which is not immediately > apparent and are hidden from cognitive regard; the same so tardy, they > herald apropos” > >

Re: Kafka behavior when consuming a topic which doesn't exist?

2017-07-07 Thread M. Manna
That depends. If auto creation of non-existent topic enabled (check docs), then it will simple use the minimum partiotion and replication settings defined in broker config to create a topic. If auto creation is disabled, your consumer group won't do anything. With auto creation enable - It's the

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-07-07 Thread Matthias J. Sax
It's too issues we want to tackle - too many overload (for some method we have already more than 10( - improve custom store API -Matthias On 7/7/17 3:42 PM, Jan Filipiak wrote: > It makes me want to cry. > > why on earth is the DSL going to expose all its implementation details now? > especi

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-07-07 Thread Jan Filipiak
It makes me want to cry. why on earth is the DSL going to expose all its implementation details now? especially being materialized or not. If we want to take usefull steps in that direction maybe we are looking for a way to let the user switch back and forth between PAPI and DSL? A change as

Kafka behavior when consuming a topic which doesn't exist?

2017-07-07 Thread Ali Akhtar
Sometimes I see warnings in my logs if i create a consumer for a topic which doesn't exist. Such as: org.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 1 : {example_topic=LEADER_NOT_AVAILABLE} If later messages are posted to that topic (which will create i

Re: Hello, Help!

2017-07-07 Thread David Garcia
“…events so timely that the bearing upon of which is not immediately apparent and are hidden from cognitive regard; the same so tardy, they herald apropos” On 7/7/17, 12:06 PM, "Marcelo Vinicius" wrote: Hello, my name is Marcelo, and I am from Brazil. I'm doing a search on Kafka. I woul

RE: Hello, Help!

2017-07-07 Thread Tauzell, Dave
As others mentioned this is not a forum to discuss the works of Franz Kafka. Here are some places to get you started: 1. The works of Franz Kafka: https://www.vanderbilt.edu/olli/class-materials/Franz_Kafka.pdf 2. Literature stack exchange: https://literature.stackexchange.com/ -Dave

Re: Hello, Help!

2017-07-07 Thread M. Manna
Is there any chance we can record this email thread in Kafka? This is Epic On 7 Jul 2017 7:15 pm, "Kim Chew" wrote: > Sorry Marcelo, Apache Kafka is a piece of computer software, you are > confused it with Franz. > > On Fri, Jul 7, 2017 at 10:06 AM, Marcelo Vinicius > wrote: > > > Hello, my na

Re: Hello, Help!

2017-07-07 Thread Kim Chew
Sorry Marcelo, Apache Kafka is a piece of computer software, you are confused it with Franz. On Fri, Jul 7, 2017 at 10:06 AM, Marcelo Vinicius wrote: > Hello, my name is Marcelo, and I am from Brazil. I'm doing a search on > Kafka. I would like to know if this phrase: "Somehow I struggled agains

Aw: Re: Broker-side Compression and Migration to lz4

2017-07-07 Thread Sven Ludwig
Hi, thanks. On another aspect in this: On a cluster that has topics with data, is it safe to gracefully shutdown a broker, change the configured compression from snappy to lz4 and start the broker? What will haben if one does that? All afterwards created new topics will come with lz4 compressio

Hello, Help!

2017-07-07 Thread Marcelo Vinicius
Hello, my name is Marcelo, and I am from Brazil. I'm doing a search on Kafka. I would like to know if this phrase: "Somehow I struggled against sensations that contained pure abstraction and no gesture directed at the present world", is it really kafka? If so, where do I find his phrase? In what te

Re: Kafka logs - server.log & controller.log

2017-07-07 Thread karan alang
sorry, missed this email Thanks for the clarification. regds, Karan Alang On Thu, Jun 29, 2017 at 6:52 AM, Tauzell, Dave wrote: > All the brokers write to server.log. The broker that happens to be the > controller will also write to the controller.log file. > > -Dave > > -Original Messag

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

2017-07-07 Thread Vahid S Hashemian
Thanks Jeff for your feedback on the usefulness of the current tool. --Vahid From: Jeff Widman To: d...@kafka.apache.org Cc: Kafka User Date: 07/06/2017 02:25 PM Subject:Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand Thanks for the KIP V

Re: Mirroring multiple clusters into one

2017-07-07 Thread Vahid S Hashemian
Thanks a lot for your input James. Regards, --Vahid From: James Cheng To: d...@kafka.apache.org Cc: users@kafka.apache.org Date: 07/06/2017 10:26 PM Subject:Re: Mirroring multiple clusters into one Answers inline below. -James Sent from my iPhone > On Jul 7, 2017, at

Re: connect in 0.11.0.0 warnings due to class not found exceptions

2017-07-07 Thread Konstantine Karantasis
Good to hear! gson is used by org.reflections Cheers, Konstantine On Thu, Jul 6, 2017 at 10:03 PM, Koert Kuipers wrote: > i did not have log4j.logger.org.reflections=ERROR, because i didnt update > my log4j files yet. i will do this now. > > connect seems to start up fine. > > i still wonder

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-07-07 Thread Guozhang Wang
I messed the indentation on github code repos; this would be easier to read: https://codeshare.io/GLWW8K Guozhang On Fri, Jul 7, 2017 at 1:30 AM, Guozhang Wang wrote: > Hi Damian / Kyle, > > I think I agree with you guys about the pros / cons of using the builder > pattern v.s. using some "s

Finding the Disk Usage of a Topic

2017-07-07 Thread Sushil Govind
Hi, I wanted to know if I can use the value of the MBean as a indicator for the Size of a topic partition.Wouldnt aggregating them for all the partitions of the Topic provide me with the disk usage per Topic ? kafka.log:type=Log,name=Size,topic= ([-.w]+),partition= ("\\d+") -- Sushil C

Re: Where to run kafka-consumer-groups.sh from?

2017-07-07 Thread Dmitriy Vsekhvalnov
I've tried 3 brokers on command line, like that: /usr/local/kafka/bin/kafka-consumer-groups.sh --bootstrap-server broker:9092,broker_2:9092,broker_3:9092 --new-consumer --group logging-svc --describe it doesn't make any difference, still x10 times difference in figures when running on broker host

Re: Where to run kafka-consumer-groups.sh from?

2017-07-07 Thread M. Manna
kafka-consumer-groups.sh --bootstrap-server broker:9092 --new-consumer --group service-group --describe how many brokers do you have in the cluster? if you have more than one, list them all using a comma csv with --bootstrap-server. Also, could you paste some results from the console printout? O

Where to run kafka-consumer-groups.sh from?

2017-07-07 Thread Dmitriy Vsekhvalnov
Hi all, question about lag checking. We've tried to periodically sample consumer lag with: kafka-consumer-groups.sh --bootstrap-server broker:9092 --new-consumer --group service-group --describe it's all fine, but depending on host we run it from it gives different results. E.g: - when runn

Re: 答复: console consumer doesn't recognise localhost:2181 but only 127.0.0.1:2181

2017-07-07 Thread M. Manna
Yes sorry that was the reason . Thanks for pointing that out :) On 7 July 2017 at 10:03, Hu Xi wrote: > “Caused by: java.net.UnknownHostException: localhsot" - localhsot? A > typo? > > > > 发件人: M. Manna > 发送时间: 2017年7月7日 16:59 > 收件人: users@kafka.apache.org

答复: console consumer doesn't recognise localhost:2181 but only 127.0.0.1:2181

2017-07-07 Thread Hu Xi
“Caused by: java.net.UnknownHostException: localhsot" - localhsot? A typo? 发件人: M. Manna 发送时间: 2017年7月7日 16:59 收件人: users@kafka.apache.org 主题: console consumer doesn't recognise localhost:2181 but only 127.0.0.1:2181 Hello, As part of my PoC I wanted to ch

console consumer doesn't recognise localhost:2181 but only 127.0.0.1:2181

2017-07-07 Thread M. Manna
Hello, As part of my PoC I wanted to check if we have two Windows 10 boxes where 1) One box will have the ZK 2) Other box will have Kafka The idea was to physically separate zookeeper and Kafka to isolate issues. For trial, I set it up on my Windows 10 machine where I used the Documentation to c

Re: [DISCUSS] Streams DSL/StateStore Refactoring

2017-07-07 Thread Guozhang Wang
Hi Damian / Kyle, I think I agree with you guys about the pros / cons of using the builder pattern v.s. using some "secondary classes". And I'm thinking if we can take a "mid" manner between these two. I spent some time with a slight different approach from Damian's current proposal: https://gith

Re: PLEASE - [KAFKA-1194] Log Retention is always throwing IOException

2017-07-07 Thread Manikumar
pl check http://kafka.apache.org/documentation.html#os On Fri, Jul 7, 2017 at 1:19 PM, M. Manna wrote: > Hello Sir, > > > Kafka is not well tested on Windows platform..There are some issues running > > on > > Windows. It is recommended to run on Linux machines. > > > *This is slightly unusual be

Re: PLEASE - [KAFKA-1194] Log Retention is always throwing IOException

2017-07-07 Thread M. Manna
Hello Sir, Kafka is not well tested on Windows platform..There are some issues running > on > Windows. It is recommended to run on Linux machines. *This is slightly unusual because I haven't seen (or perhaps, unintentionally overlooked) any documentation/blogs which states this.Any Java library