Re: Kafka 0.8 producer: Failed to send messages after 3 tries

2013-06-18 Thread Markus Roder
Ok, then it seems as I do not have the latest API, as if I'm using "metadata.broker.list" I got the error " requirement failed: Missing required property 'broker.list' " Let me check if I can figure out on my side, what is going wrong. Thanks for your hint. regards Markus 2013/6/18 Jun Rao > A

Re: Kafka 0.8 producer: Failed to send messages after 3 tries

2013-06-18 Thread Markus Roder
Hi Jun, thanks for your hint, with the newest 0.8beta1 code it is now working. regards Markus 2013/6/18 Markus Roder > Ok, then it seems as I do not have the latest API, as if I'm using > "metadata.broker.list" I got the error " requirement failed: Missing > required property 'broker.list' " >

Fwd: IMPORTANT: Major Confluence Upgrade Coming Soon. Please review test instance now.

2013-06-18 Thread Joe Stein
FYI in regards to https://cwiki.apache.org/confluence/display/KAFKA/Index -- Forwarded message -- From: gmcdonald Date: Tue, Jun 18, 2013 at 7:56 AM Subject: IMPORTANT: Major Confluence Upgrade Coming Soon. Please review test instance now. To: committ...@apache.org [Note this me

Partition data for deleted topic found in kafka-logs, also, found leader: -1

2013-06-18 Thread Yogesh Sangvikar
Hi Team, I am exploring kafka 0.8 beta release to understand data flow, replication features. While testing i found that, the partition data for data for deleted topic is preseved in kafka-logs, why this behavior? suppose below case, A topic (suppose test1) is created with partition 6 and replic

kafka source code run problems

2013-06-18 Thread sphinx jiang
Hi, I am Sphinx Jiang, a Chinese Kafka user. I am learning the source code. But after I import the source code in eclipse, the running come with some errors. Now the problem is like this: [image: 内嵌图片 1] I just wondering as source code, the error must be caused by my configuration, right? Is ther

Facebook like newsfeed using kafka

2013-06-18 Thread Piyush Rai
I am trying to build a news feed similar to facebook in which user can subscribe set of items and topics and they will receive feeds as that subscribed topic changes. I was looking around and many people say kafka is good choice. Can kafka be self sufficient to build the entire news feed or should

Subscribe me

2013-06-18 Thread Dibyendu Bhattacharya
Regards, Dibyendu

C# client for kafka 0.8

2013-06-18 Thread Withers, Robert
I see an old C# client, which is 2 years old. Does anyone have a C# client that works with the kafka 0.8 producer? Thanks, rob

Re: Facebook like newsfeed using kafka

2013-06-18 Thread Philip O'Toole
You should consider using it regardless. I find 0.72 to be a great system, which is well designed and reliable. As for Storm, it depends. If you just want a simple pub-sub queue system, probably not. Philip On Jun 18, 2013, at 6:48 AM, Piyush Rai wrote: > I am trying to build a news feed

Re: C# client for kafka 0.8

2013-06-18 Thread Timothy Chen
Hi Robert, The most recent one that I know of is the C# client that ExactTarget folks did, however not all calls are up to the 0.8 protocol so it doesn't completely work. I have a slightly more cleaned up version here https://github.com/tnachen/kafka/tree/feature/et-develop-0.8 It will be great

Re: Upgrade to Zookeeper 3.4.5 stable ?

2013-06-18 Thread Jun Rao
We do plan to look into upgrading to ZK 3.4.5 at some point, especially for picking up the multi() feature. ZK 3.4.5 should be backward compatible. Do you know how stable it is? Thanks, Jun On Mon, Jun 17, 2013 at 9:36 PM, Ran RanUser wrote: > I'm running Zookeeper 3.4.5 server and using 3.4.

why was auto.offset.reset default changed (so late in the game)?

2013-06-18 Thread Jason Rosenberg
I'm wondering why the default setting for auto.offset.reset in the ConsumerConfig class was changed from 'smallest' to 'largest', so late in the game (looks like a commit on June 3 changed the default). This is an extremely major change, I should think. Consumers now by default only get messages

Re: why was auto.offset.reset default changed (so late in the game)?

2013-06-18 Thread Jun Rao
Hi, Jason, The reasons for this change are: (1) This is probably the most convenient setting for people to migrate from 0.7 to 0.8. The process is to build an 0.8 shadow cluster using our migration tool, upgrade all consumers to 0.8, and finally upgrade all producers to 0.8. Since most consumers a

Re: why was auto.offset.reset default changed (so late in the game)?

2013-06-18 Thread Jason Rosenberg
Jun, I just think it's a major change, that might have warranted a heads up, but it's all good (I've got things looking good with it now). I simply changed the default behavior everywhere to explicitly initialize consumers with 'smallest' by default (whereas previously this was an ignored config

Re: why was auto.offset.reset default changed (so late in the game)?

2013-06-18 Thread Jun Rao
Jason, If we default to smallest and a consumer doesn't override this, when it migrates to 0.8, it will likely reconsume a lot of data. Quite of few consumers are real time since they feed data to systems like Storm. What do you think is the best way to communicate such config changes in the futu

Re: Partition data for deleted topic found in kafka-logs, also, found leader: -1

2013-06-18 Thread Yogesh Sangvikar
Hi Team, Could you please help to understand those features? Thanks, Yogesh Sangvikar On Tue, Jun 18, 2013 at 6:55 PM, Yogesh Sangvikar < yogesh.sangvi...@gmail.com> wrote: > Hi Team, > > I am exploring kafka 0.8 beta release to understand data flow, replication > features. > While testing i f

0.8 beta already deploy in production environment

2013-06-18 Thread Markus Roder
Hi kafka-users and developers, we are currently considering using kafka 0.8 in production environment. Is this a no-go in the current state of development or do you think we can give it the risk already? How do you think is the near future timeline of kafka 0.8? When do you expect it gets stable?