Re: could an Encoder/Decoder be stateful?

2013-05-15 Thread Jun Rao
Each producer/consumer uses a single instance of the encoder/decoder. Thanks, Jun On Wed, May 15, 2013 at 7:20 PM, Rob Withers wrote: > Or is the same instance used for each (un)marshaling? It would be nice to > have a cache and a duplicateMsgChecker function, from the app above to > ensure

Re: API to to query messages amount under one topic

2013-05-15 Thread Jun Rao
In 0.8, you can get the earliest and the latest offset using the getOffsetBefore api. The difference btw the two gives the number of messages on the broker. Thanks, Jun On Wed, May 15, 2013 at 6:05 PM, Sining Ma wrote: > Hi, > Is there any APIs in kafka that I can use to query how many messag

Re: Capacity planning for kafka

2013-05-15 Thread Jun Rao
In general, Kafka brokers are low in CPU, memory and I/Os. We do rely on the broker server to cache all recent data in pagecache. The biggest contraint is often the disk space, especially if you keep the default retention time for 7 days. From this perspective, HDDs are better than SSDs since the p

Re: when do you think 0.8 could get promoted from beta to a release?

2013-05-15 Thread Joe Stein
That would be awesome! I will have cycles the last week of May and first week of June to help prepare the artifacts for release vote once the vote passes I could push to a maven repo On Thu, May 16, 2013 at 12:25 AM, Neha Narkhede wrote: > We are currently working on fixing a blocker perform

Re: when do you think 0.8 could get promoted from beta to a release?

2013-05-15 Thread Neha Narkhede
We are currently working on fixing a blocker performance issue (KAFKA-901). We expect to resolve it by early next week after which a beta will be available. We could maybe target the release mid June? Thanks, Neha On May 15, 2013 7:27 PM, "Rob Withers" wrote: > We are curious. It would be excel

when do you think 0.8 could get promoted from beta to a release?

2013-05-15 Thread Rob Withers
We are curious. It would be excellent if around 8/17 could be targeted, perhaps go for 7/17 as RC and let 8/17 be a RC2 date, a month before we would like to see it go to production. An RC with thorough testing with our application may be workable. Thanks, rob

could an Encoder/Decoder be stateful?

2013-05-15 Thread Rob Withers
Or is the same instance used for each (un)marshaling? It would be nice to have a cache and a duplicateMsgChecker function, from the app above to ensure transactional guarantees, and object ref substitutions during (de)serialization, to enable durable distributed objects and promises. Thanks,

RE: API to to query messages amount under one topic

2013-05-15 Thread Rob Withers
+ 1! A call through the Consumer Group, queuedMessageCount(topic, partitionNumber), could query zookeeper (and add the local fetched queue) to return this interesting piece of information. That is if we cannot already do this :) Thanks, rob > -Original Message- > From: Sining Ma [mailto

Re: LeaderNotAvailableException

2013-05-15 Thread Andrea Gazzarini
Hi, don't know exactly can help but I got the same LeaderNotAvailableException while playing (several restarts) with different versions of Zookeeper pointing to the same dataDir (without cleaning that between restarts) Regards, Andrea On 05/14/2013 06:12 PM, Yu, Libo wrote: Hi, I have 3 mac