Re: [ANNOUNCE] Apache Kafka 0.11.0.3 Released

Thanks for driving the release, Matthias! On Tue, Jul 3, 2018 at 10:08 PM, Jason Gustafson wrote: > Awesome. Thanks Matthias! > > On Tue, Jul 3, 2018 at 12:44 PM, Yishun Guan wrote: > > > Nice! Thanks~ > > > > On Tue, Jul 3, 2018, 12:16 PM Ismael Juma wrote: > > > > > Thanks Matthias! > > > >

Re: [ANNOUNCE] Apache Kafka 0.10.2.2 Released

Thanks for driving the release, Matthias! On Tue, Jul 3, 2018 at 8:48 PM, Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > The Apache Kafka community is pleased to announce the release for > Apache Kafka 0.10.2.2. > > > This is a bug fix release and it includes fixe

Kafka performance tests

Hi, I am working on testing the performance of Kafka. I am looking for articles, publications about performance tests, methods, results and configurations, but so far I have not found many. Do you know any useful studies? Has any of you done something similar? If yes, would you mind sharing those

Re: Kafka performance tests

Hi I am working on this too and I still try to search nice sources how to optimize results. I can recommend you this article Benchmarking and this presentation

Kafka disk recommendation for cloud

Hi All, I was wondering what the recommendations are for disk type for hosting Kafka on a cloud environment? As far as I know, most of the best practices suggest using spinning disks for Kafka due to the fact that Kafka architecture relies on sequential write/read. Hence, the increase in Kafka per

Brokers are failing to handle UpdateMetadata and LeaderAndIsRequest requests from controller

Hi, I have a Kafka 1.1 cluster that doesn't start up correctly: many partitions are not replicated correctly. In the controller.log I see messages indicating that brokers aren't succesfully handling LeaderAndIsRequest and UpdateMetadata requests. For example: [2018-07-04 13:31:18,788] WARN [Req

Offset reprocess loop over

Hello all, we have in our infrastructure, 3 brokers. Sometimes we order a "reprocess all" certain flow, but we are facing a problem ... After relaunch reprocessing at the beginning offset, and arrived at a number offset, it loops several times by returning to an previous offset. for example,

Processor API, how to get last N hours word count

Hello, Just started with Kafka, took 2.0 because it has better unit test support. Built custom processor, which is basically same as WordCountProcessorDemo example (github https://goo.gl/XSh7iW ) and can be treated as equal to that. Built topology by adding source topic, processor, state store, an

Re: Kafka Streams - Shared Ktable State Store

Hello, Thanks for sharing your use scenarios, this seems a common multi-DC deployment question where you want to have a smooth process upon Kafka cluster failover. I'd like to ask where is your Streams application sitting? Is it sitting in one of the data centers or it is sitting in another differ

Re: Processor API, how to get last N hours word count

Hello Gleb, For the first question, you should use a windowed store in your topology builder. And for the first / second question, I think using interactive query should be fine, i.e. you can create different windows for different length, and use interactive queries to get the count on different

Re: Processor API, how to get last N hours word count

Hello Guozhang, Thank you! One thing to clarify: so, only persistent store can be windowed, right? Demo default in-memory key-value store won't work? /G. On Wed, Jul 4, 2018 at 9:45 PM, Guozhang Wang wrote: > Hello Gleb, > > For the first question, you should use a windowed store in your topolog