RE: Problem with node after restart no partitions?

2015-04-01 Thread Thunder Stumpges
Well it appears we lost all the data on the one node again. It appears to be all or part of KAFKA-1647 as we saw this in our logs (for all topics): [2015-04-01 10:46:58,901] WARN Partition [logactivity-redirect,3] on broker 6: No checkpointed hi

Re: blocking KafkaProducer call

2015-04-01 Thread Mayuresh Gharat
Great !!. The sync mode and those properties are orthogonal. Thanks, Mayuresh On Wed, Apr 1, 2015 at 10:46 AM, sunil kalva wrote: > thanks grant > by changing linger.ms=0,batch.size=1, problem solved > > > On Wed, Apr 1, 2015 at 11:11 PM, Grant Henke wrote: > > > They can. You can read more a

Re: blocking KafkaProducer call

2015-04-01 Thread sunil kalva
thanks grant by changing linger.ms=0,batch.size=1, problem solved On Wed, Apr 1, 2015 at 11:11 PM, Grant Henke wrote: > They can. You can read more about configuring the new java producer here: > http://kafka.apache.org/documentation.html#newproducerconfigs > > Thanks, > Grant > > On Wed, Apr 1

Re: blocking KafkaProducer call

2015-04-01 Thread Grant Henke
They can. You can read more about configuring the new java producer here: http://kafka.apache.org/documentation.html#newproducerconfigs Thanks, Grant On Wed, Apr 1, 2015 at 12:34 PM, sunil kalva wrote: > Does these config params has effect when i try to simulate "sync" mode by > not passing cal

Re: blocking KafkaProducer call

2015-04-01 Thread sunil kalva
Does these config params has effect when i try to simulate "sync" mode by not passing callback ? On Wed, Apr 1, 2015 at 10:32 PM, Mayuresh Gharat wrote: > Whats your "linger.ms" and "batch.size" ? > > Thanks, > > Mayuresh > > On Wed, Apr 1, 2015 at 5:51 AM, sunil kalva wrote: > > > I am trying

Problem with node after restart no partitions?

2015-04-01 Thread Thunder Stumpges
Hi group, We've been trying to track down a problem this morning for a little while, and thought I'd ask here while we keep looking. We have 3 nodes (rep-3) running 8.1.1. We attempted a rolling upgrade yesterday to 8.2.1, and on the first node, after restarting, a single topic (a samza interm

Re: Consumer in Java client

2015-04-01 Thread Mayuresh Gharat
It will have an API for committing offsets. We are planning to provide functionality with separate client for fetching offsets and Consumer may or may not use that client. Thanks, Mayuresh On Tue, Mar 31, 2015 at 3:16 PM, Hema Bhatia wrote: > 2 more questions to add to it: > > - Any estimation

Re: blocking KafkaProducer call

2015-04-01 Thread Mayuresh Gharat
Whats your "linger.ms" and "batch.size" ? Thanks, Mayuresh On Wed, Apr 1, 2015 at 5:51 AM, sunil kalva wrote: > I am trying to simulate "sync" call using following code, > > try { > > Future send = producer.send(new > ProducerRecord("the-topic", "key".getBytes(), > "value".getBytes())).get

Re: Which version works for kafka 0.8.2 as consumer?

2015-04-01 Thread Mayuresh Gharat
What do you mean by offset syncing? Thanks, Mayuresh On Wed, Apr 1, 2015 at 9:59 AM, pushkar priyadarshi < priyadarshi.push...@gmail.com> wrote: > So in 0.8.2.0/0.8.2.1 high level consumer can not make use of offset > syncing in kafka? > > On Wed, Apr 1, 2015 at 12:51 PM, Jiangjie Qin > wrote:

Re: Which version works for kafka 0.8.2 as consumer?

2015-04-01 Thread pushkar priyadarshi
So in 0.8.2.0/0.8.2.1 high level consumer can not make use of offset syncing in kafka? On Wed, Apr 1, 2015 at 12:51 PM, Jiangjie Qin wrote: > Yes, KafkaConsumer in 0.8.2 is still in development. You probably still > want to use ZookeeperConsumerConnector for now. > > On 4/1/15, 9:28 AM, "Mark Za

Re: Which version works for kafka 0.8.2 as consumer?

2015-04-01 Thread Jiangjie Qin
Yes, KafkaConsumer in 0.8.2 is still in development. You probably still want to use ZookeeperConsumerConnector for now. On 4/1/15, 9:28 AM, "Mark Zang" wrote: >I found the 0.8.2.0 and 0.8.2.1 has a KafkaConsumer. But this class seems >not completed and not functional. Lots of method returns null

Which version works for kafka 0.8.2 as consumer?

2015-04-01 Thread Mark Zang
I found the 0.8.2.0 and 0.8.2.1 has a KafkaConsumer. But this class seems not completed and not functional. Lots of method returns null or throws NSM. Which version of consumer for kafka 0.8.2 broker? Thanks! -- Best regards! Mike Zang

Re: blocking KafkaProducer call

2015-04-01 Thread sunil kalva
I am trying to simulate "sync" call using following code, try { Future send = producer.send(new ProducerRecord("the-topic", "key".getBytes(), "value".getBytes())).get(); send.get(); System.out.println("Time = " + (System.currentTimeMillis() - b)); } catch (Exception e) { } And i