Re: Async producer using Sync producer for send

Thanks Jiangjie, I too have thought the same after looking the code. Thanks a lot for clearing my doubt! On Tue, Mar 31, 2015 at 11:45 AM, Jiangjie Qin wrote: > The async send() put the message into a message queue then returns. When > the messages are pulled out of the queue by the sender thre

Re: Async producer using Sync producer for send

The async send() put the message into a message queue then returns. When the messages are pulled out of the queue by the sender thread, it still uses SyncProducer to send ProducerRequests to brokers. Jiangjie (Becket) Qin On 3/30/15, 10:44 PM, "Madhukar Bharti" wrote: >Hi All, > >I am using *as

Re: Consumer in Java client

Hi, KafkaConsumer is still under development and not ready for wide use case. Currently, it can be used to replace SimpleConsumer (low level consumer), but can not replace ZookeeperConsumerConnector(high level consumer). So if you need to use simple consumer, I would suggest using KafkaConsumer in

Async producer using Sync producer for send

Hi All, I am using *async *producer to send the data. When I checked the log it is showing as below: [2015-03-31 11:09:55,915] INFO Verifying properties (kafka.utils.VerifiableProperties) [2015-03-31 11:09:55,946] INFO Property key.serializer.class is overridden to kafka.serializer.StringEncoder

Consumer in Java client

Hi I'm new to kafka and using 0.8.2.1 kafka-clients and kafka_2.10 packages. The document says: “We are in the process of rewritting the JVM clients for Kafka. As of 0.8.2 Kafka includes a newly rewritten Java producer. The next release will include an equivalent Java consumer.” It seems the kafka

Re: Kafka server relocation

Do you have data sending to *testtopic? *By default mirror maker only consumes data being sent after it taps into the topic. you need to keep sending data to the topic after mirror maker connection is established. If you want to change the behavior you can set auto.offset.reset=smallest so that any

Re: Kafka server relocation

thanks Xiao I tried MirrorMaker option in my test environment but failed. I am not able to see the log getting copied to destination cluster. I see in the log of MirrorMaker process that connection is successfully established between source and destination cluster but still not sure what i causing

Re: GC problem on broker startup under minimal load

Sorry - some final details on this for today. I stopped the broker being added to the cluster (the one with GC taking 1min & both cores), dropped the max heap to 1GB (out of 4GB) and started it up again. It joined nicely, caught up and went ISR. Haven't yet run into the increasing GC cycles (which

New kafka client for Go (golang)

Hi, I wanted to share new client library for Go language that I'm developing at Optiopay. Library is MIT licensed and provides API close to latest kafka request/response wire format with few higher level helpers to make common tasks easier. The only thing that is not yet implemented is message set

Re: GC problem on broker startup under minimal load

A few more details before the looming crash. The broker replicating data and in sync (before GC cycle starts) - GC cycles taking nearly 1min real: 2015-03-30T21:08:39.502+0300: 3805.931: [GC2015-03-30T21:08:39.502+0300: 3805.932: [ParNew: 142199K->6219K(153344K), 51.6581120 secs] 678471K->543078K(

Re: GC problem on broker startup under minimal load

Hi, A related issue which results in a similar increasing GC cycles (but doesn't recover)... We're trying to bootstrap a new broker into the cluster (replacing one of the 3 main ones described previously). It connects fine, starts replicating data and becomes ISR, but starts a similar GC cycle on

Re: Java NPE in using KafkaConsumer API

Are you using the 0.8.2 release or trunk? -Jay On Mon, Mar 30, 2015 at 1:35 AM, Sandeep Bishnoi < sandeepbishnoi.b...@gmail.com> wrote: > Hi, > > I have configured a kafka consumer as follows: > Properties props = new Properties(); > // .. > // Populated properties > Kafka

Re: Fwd: Java NPE in using KafkaConsumer API

On 03/30/2015 04:35 AM, Sandeep Bishnoi wrote: > > The consumer.poll() api is returning null always. Although the remaining > code is based on the returned value from poll. > > Can you please let me know whether > org.apache.kafka.clients.consumer.KafkaConsumer is a valid API to use if I > wa

Re: using 0.8.2 in production

The latest available version is 0.8.2.1 Download it from here One of our teams have moved to this version in production and have reported no issues. On Mon, Mar 30, 2015 at 7:24 PM, pushkar priyadarshi < priyadarshi.push...@gmail.com> wrote: > Hi, > > I r

Fwd: Java NPE in using KafkaConsumer API

Hi, I have configured a kafka consumer as follows: Properties props = new Properties(); // .. // Populated properties KafkaConsumer consumer = new KafkaConsumer(props); // subscribe to partition "0" of topic test TopicPartition partition0 = new TopicPartition("test"

using 0.8.2 in production

Hi, I remember some time back people were asked not to upgrade to 0.8.2.Wanted to know if issues pertaining to that are resolved now and is it safe now to migrate to 0.8.2? Thanks And Regards, Pushkar