message order problem

Broker setting is: 8 partitions, 1 replica, kafka version 0.8.1 We send 2 message at almost same time. Msg1 first, Msg2 second. We have more than 1 producers in sync mode. We may send msg1 in one broker, after *producer.send return response* And send msg2 in the other broker. Both 2 msg has

Re: Updated protocol guide for 0.9

Thanks Jason! The updated doc looks very good to me. I've spent some time reading through the new APIs and the Proposal doc and everything seems pretty clear so far. This really helps, thanks again! Also, +1 on putting this doc under version control so that when there are any changes to the prot

Re: Updated protocol guide for 0.9

Hi Anatoly, I spent a little time this afternoon updating the request types and error codes. This wiki is getting a little difficult to manage, especially in regard to error codes, so I opened KAFKA-2865 to hopefully improve the situation. Probably we need to pull this documentation into the proje

JVM parameters for the Kafka consumer

Hi everyone, In the Kafka documentation under the *Java Version* section it is recommended to use JDK 1.7 u51 and the G1 garbage collector for brokers. - Is that the same for the high level consumers? - For JDK1.8, which update and what JVM parameters are recommended for the consumers as well?

Re: Partial broker shutdown causing producers to stall even with replicas

I've logged this as https://issues.apache.org/jira/browse/KAFKA-2864 -- Mahdi. On 11/16/15 12:59 PM, Rajiv Kurian wrote: Yes I think so. We specifically upgraded the Kafka broker with a patch to avoid the ZK client NPEs. Guess not all of them are fixed. The Kafka broker becoming a zombie even i

Re: Release of 0.9.0

I think RC3 is getting voted on so it's close. Kafka I don't think the Kafka community ever has "release dates". It's more that RC's get voted on and then if they are good, release happens. So just watch for RC votes. On Thu, Nov 19, 2015 at 1:16 PM, Mohit Anchlia wrote: > Is there a tentative d

Possible to easily translate a high level consumer consumption process to simple consumer when consuming by offset is needed?

Scenario: A kafka + zookeeper setup with topics that span partitions. Obviously high level consumer is desirable here. But let's say we have a need to consume by offset, and we don't want the earliest offset as that has extra messages we'd have to filter out, and from latest offset doesn't have t

Updated protocol guide for 0.9

Hi, Does anyone know if there is updated version of this page https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol for the upcoming release of 0.9? Best, Anatoly

Re: [VOTE] 0.9.0.0 Candiate 3

+1 (non-binding). Verified source and binary artifacts, ran ./gradlew testAll with JDK 7u80, quick start on source artifact and Scala 2.11 binary artifact. Ismael On Wed, Nov 18, 2015 at 5:57 AM, Jun Rao wrote: > This is the third candidate for release of Apache Kafka 0.9.0.0. This a > major r

Release of 0.9.0

Is there a tentative date for the release of 0.9.0? I tried looking at Jira tickets however there is no mention of a tentative date when 0.9.0 is going to be released.

offset consume in JAVA api

Hi, I found in 0.9, def commitOffsets(offsetsToCommit: java.util.Map[TopicAndPartition, OffsetAndMetadata], retryOnFailure: Boolean) is added to ZookeeperConsumerConnector. If I want to read data backwards or forwards, can I use this API to commit new offset and then read data from new offset?

Re: consumer offset tool and JMX metrics do not match

Anyone can help me understand this? On Mon, Nov 16, 2015 at 11:21 PM, allen chan wrote: > According to documentation, offsets by default are committed every 10 > secs. Shouldnt that be frequent enough that JMX would be accurate? > > autocommit.interval.ms1is the frequency that the consumed o