Re: 0.8.1 Java Producer API Callbacks

This summary is correct. If you are just starting development now it is probably reasonable to start with the new producer. We would certainly appreciate any feedback on it. My recommendation would be to build the producer off trunk as there were a few bug fixes since 0.8.1.x that are worth gettin

Re: JAVA HEAP settings for KAFKA in production

Hi, We have 2 small EC2 instances acting as "Receivers" for SPM. Each of them runs Apache, Jetty, and a Kafka broker with the following params: java -Xmx512M -server -Dlog4j.configuration=file:/usr/lib/kafka/bin/../config/log4j.properties -cp :/usr/lib/kafka/bin/../proj

Re: 0.8.1 Java Producer API Callbacks

Hi Christian, As Jun mentioned, the new producer is marked beta since it is new. By 0.9 or even sooner, we'd expect it to be deployed at LinkedIn and a few other companies in a stable state. The APIs, guarantees and features will not change by 0.9, just the stability. Thanks, Neha On Fri, May 2

Re: Kafka Consumer consuming half message

Some of the FAQs for the consumers should help you investigate this. In particular, take a look at the ConsumerOffsetChecker tool in https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped,why ? Thanks, Jun On Fri, May 2, 2014 at 5:07 AM, naveen wrote: > Hi, > >

Re: command line clients and request balancing.

The reason for this change is described in https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified ? In 0.8.2, we will be releasing the new producer which will have the even distribution out of the box, but allow the

Re: 0.8.1 Java Producer API Callbacks

It's beta mostly because it's new. Thanks, Jun On Thu, May 1, 2014 at 10:25 PM, cac...@gmail.com wrote: > Thanks, that's quite helpful. According to this post, > http://blog.empathybox.com/ , it looks like it will be beta then which > seems good enough. Assuming that the beta designation is c

Re: log.retention.size

log.retention.size controls the total size in a log dir (per partition). log.file.size controls the size of each log segment in the log dir. Thanks, Jun On Thu, May 1, 2014 at 9:31 PM, vinh wrote: > In the 0.7 docs, the description for log.retention.size and log.file.size > sound very much th

building consumers with Kafka 0.8.1 but POM (Maven build tool) not working

I am creating some consumers and producers to test against some Kafka queues. I am getting this error when I try to build or test the code using Maven: -- [ERROR] Failed to execute goal on project Kafka_test: Could not resolve dependencies for project com.cat:Kafka_test:jar:1.0-SNAPSHOT: The follo

Kafka Consumer consuming half message

Hi, My consumer process has configured for 3 zookeper url and i am running this process for 10 topics . But some time i get only half message for some topic. am i missing something or broker is sending half message only. please let me know if u need any more information. Thanks, Naveen Kumar

Re: command line clients and request balancing.

Why was this change made? Does this mean that all applicationa i have will need to be converted to use the partitioned send? This hurts the out of the box experience. Can the console producer simply split on first space and use that as the key? On Thursday, May 1, 2014, Guozhang Wang wrote: > Hi