Re: Truncated messages in fetch response

2013-06-26 Thread Jay Kreps
Yeah, that is true. I thought I documented that, but looking at the protocol docs, it looks like I didn't. I agree this is kind of a pain in the ass. It was an important optimization in 0.7 because we didn't know where the message boundaries were but in 0.8 we have a fast way to compute message bo

Truncated messages in fetch response

2013-06-26 Thread Bob Potter
Howdy, I'm developing a client for kafka 0.8. It looks like a fetch response will sometimes end with a partial message. I understand why this might be the case but it was unexpected and as far as I can tell undocumented. Is my understanding correct or am i missing something? -Bob

Kafka User Group Meeting Jun. 27

2013-06-26 Thread Jun Rao
Hi, Everyone, We have finalized our agenda of the meetup Thursday evening, with Speakers from LinkedIn, RichRelevance, Netflix and Square. Please RSVP to the meetup link below. For remote people, we will post the streaming link at meetup before the meeting and we will use our IRC for remote questi

Re: Troubles with ZK 3.3.5 - What do you run in prod?

2013-06-26 Thread Jun Rao
At LinkedIn, we are running ZK 3.3.4. Does that issue happen to 3.3.4 too? Thanks, Jun On Wed, Jun 26, 2013 at 2:48 AM, Alexandre Rodrigues < alexan...@blismedia.com> wrote: > Hi guys, > > I am having trouble running with ZK 3.3.5 due to > https://issues.apache.org/jira/browse/ZOOKEEPER-1546 .

Re: Setting up kafka consumer to send logs to s3

2013-06-26 Thread Glenn Nethercutt
I did the same search about a month ago. I forked this kafka-s3-consumer from QubitProducts/kafka-s3-consumer to upfit it for the 0.8 api changes. You'll have to futz with th

Re: Troubles with ZK 3.3.5 - What do you run in prod?

2013-06-26 Thread Philip O'Toole
3.4.5. Philip On Jun 26, 2013, at 2:48 AM, Alexandre Rodrigues wrote: > Hi guys, > > I am having trouble running with ZK 3.3.5 due to > https://issues.apache.org/jira/browse/ZOOKEEPER-1546 . What version of ZK > do you run in production environments? > > Thanks, > Alex > > -- > > @BlisMe

Re: Kafka Responsiveness

2013-06-26 Thread Hanish Bansal
Thanks for reply :) On Mon, Jun 24, 2013 at 5:45 PM, Florin Trofin wrote: > This might work OK for 0.7 but you might run into trouble with 0.8 when > replication is enabled. Make sure you test all the different scenarios for > failure. See the previous discussion thread "Kafka 0.8 Failover Beha

Troubles with ZK 3.3.5 - What do you run in prod?

2013-06-26 Thread Alexandre Rodrigues
Hi guys, I am having trouble running with ZK 3.3.5 due to https://issues.apache.org/jira/browse/ZOOKEEPER-1546 . What version of ZK do you run in production environments? Thanks, Alex -- @BlisMedia www.blismedia.com This email and any att

Re: kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.

2013-06-26 Thread Yogesh Sangvikar
Hi Team, We found the issue. It is due to missing method 'com.yammer.metrics.core.TimerContext.stop()' in the metrics jar that we are using. We have downgrading the metrics jar version from 3.x to 2.2.0 and found producer is working as expected. Thanks for the help team. Thanks, Yogesh Sangvikar