Re: Lots of messages when starting Kafka

2016-07-06 Thread Peter Davis
al Ctrl+C stops Kafka way faster than 10 seconds. > > > On 06/07/16 16:05, Peter Davis wrote: > > You're right Sam. Getting my signals confused before coffee. > > > > Fabian, how did you in fact kill Kafka? "kill"? Ctrl+C? > > > > -Peter >

Re: Lots of messages when starting Kafka

2016-07-06 Thread Peter Davis
close the contents to another person, use it for any purpose, or store, > or copy > the information in any medium. Please also destroy and delete the message > from > your computer. > > > On 6 July 2016 at 14:54, Peter Davis wrote: > > > If you kill Kafka with SIGT

Re: Lots of messages when starting Kafka

2016-07-06 Thread Peter Davis
If you kill Kafka with SIGTERM, then naturally it will leave behind corrupt index files. It will safely rebuild them. Don't worry. The Group Metadata Manager messages are normal -- that's just Kafka starting up. Perhaps the corrupted index log messages could be less scary, though. This is a very

Re: Producer Properties

2016-07-01 Thread Peter Davis
Chris, all of the keys are documented here: http://kafka.apache.org/documentation.html#producerconfigs Easier to read than ProducerConfig? -Peter > On Jun 27, 2016, at 6:34 AM, Chris Barlock wrote: > > Anybody? > > Chris > > > > > From: Chris Barlock/Raleigh/IBM@IBMUS > To: users@

Re: console-consumer show offset

2016-07-01 Thread Peter Davis
Vincent, you'll need to provide a custom Message Formatter implementation. I too found myself wishing for this for debugging purposes, but the default one doesn't support it. > ./bin/kafka-console-consumer --new-consumer --bootstrap.server localhost:9092 > --topic foo --formatter my.custom.Me

Re: Brokers are crash due to __consumer_offsets folder are deleted

2016-07-01 Thread Peter Davis
Dear 黄杰斌: I am guessing your operating system is configured to delete your /tmp directory when you restart the server. You will need to change the "log.dir" property in your broker's server.properties file to someplace permanent. Unfortunately, your data is lost unless you had a backup or had

Re: Halting because log truncation is not allowed for topic __consumer_offsets

2016-06-26 Thread Peter Davis
ring normal operation as well. > > -James > > Sent from my iPhone > > > On Jun 26, 2016, at 1:41 PM, Peter Davis wrote: > > > > We have seen this several times and it's quite frustrating. It seems to > happen due to the fact that the leader fo

Re: Halting because log truncation is not allowed for topic __consumer_offsets

2016-06-26 Thread Peter Davis
We have seen this several times and it's quite frustrating. It seems to happen due to the fact that the leader for a partition writes to followers ahead of committing itself, especially for a topic like __consumer_offsets that is written with acks=all. If a brief network interruption occurs (a

Re: WordCount example errors

2016-06-14 Thread Peter Davis
This looks like the error that occurs when you use the 0.10 client with an 0.9 broker. The broker needs to be upgraded first. Jeyhun, what versions are you running? (I sincerely hope this error message will be improved next time there are wire protocol changes!) -Peter > On Jun 14, 2016, a

Re: Connect MemoryOffsetBackingStore creates a non-daemon thread, prevents clean shutdown

2016-05-13 Thread Peter Davis
Created https://issues.apache.org/jira/browse/KAFKA-3710 Regards, Peter Davis (206) 713-7352 davi...@gmail.com On Fri, May 13, 2016 at 7:02 AM, Peter Davis wrote: > I tried. JIRA is locked down. > > -Peter > > > > On May 12, 2016, at 18:14, Liquan Pei wrote: > >

Re: Connect MemoryOffsetBackingStore creates a non-daemon thread, prevents clean shutdown

2016-05-13 Thread Peter Davis
I tried. JIRA is locked down. -Peter > On May 12, 2016, at 18:14, Liquan Pei wrote: > > Hi Peter, > > That seems to be an issue. Do you want to file a JIRA on this? > > Thanks, > Liquan > >> On Thu, May 12, 2016 at 5:01 PM, Peter Davis wrote: >> &

Connect MemoryOffsetBackingStore creates a non-daemon thread, prevents clean shutdown

2016-05-12 Thread Peter Davis
I'm having an issue running a Connect source worker running as a standalone herder embedded in a webapp. The webapp hangs on shutdown. MemoryOffsetBackingStore creates an executor but doesn't call ExecutorService.shutdown() on stop(). I'd like to contribute a pull request. Question in the meant

Re: compacted log TTLs

2016-05-12 Thread Peter Davis
One use case is implementing a data retention policy. -Peter > On May 12, 2016, at 17:11, Guozhang Wang wrote: > > Wesley, > > Could describe your use case a bit more for motivating this? Is your data > source expiring records and hence you want to auto "delete" the > corresponding Kafka reco