how can i use offset filter the msg

2013-10-20 Thread kojie . fu
Hi all, we have a scene like this: A system produce the msg ,B system comsumer the msg. for some reason the B system unwork,and then we need abandon the out of date msg. but now in our msg there is no time stamp. how can i use the offset to do this. does kafka know when the msg produce? how ca

[jira] [Reopened] (KAFKA-1042) Fix segment flush logic

2013-10-20 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reopened KAFKA-1042: Trunk has a compilation error after this patch is checked in. [error] /Users/jrao/Intellij/kafka_git/core/sr

Re: Review Request 14771: Log.getOffsetsBefore(t, …) does not return the last confirmed offset before t

2013-10-20 Thread Jun Rao
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14771/#review27228 --- Thanks for the patch. The issue is that firstAppendTime is no longer

Re: preparing for the 0.8 final release

2013-10-20 Thread Jun Rao
The fetch thread uses multi-fetch. Have you looked at https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whymessagesaredelayedinmyconsumer%3F, which may be related to your issue. Thanks, Jun On Sun, Oct 20, 2013 at 2:48 AM, Jason Rosenberg wrote: > Ok, > > So here's an outline of what

Re: preparing for the 0.8 final release

2013-10-20 Thread Jason Rosenberg
Ok, So here's an outline of what I think seems to have happened. I have a consumer, that uses a filter to consume a large number of topics (e.g. several hundred). Each topic has only a single partition. It normally has no trouble keeping up processing all messages on all topics. However, we h