RE: Reg Exception in Kafka

I don't think so. I am able to connect to the server using a putty. It is VM running in Amazon cloud. Thanks Bala -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, 19 February, 2014 12:32 PM To: users@kafka.apache.org Subject: Re: Reg Exception in Kafka Any is

Re: Surprisingly high network traffic between kafka servers

Ok, I filed a jira for that.https://issues.apache.org/jira/browse/KAFKA-1273 -- Original -- From: "Jun Rao"; Date: 2014年2月14日(星期五) 晚上11:54 To: "users@kafka.apache.org"; Subject: Re: Surprisingly high network traffic between kafka servers Hi, Zhong, Thanks

Partitions and Scale out

Hello Masters, I have 2 questions: 1. Whether all Topic's partitions include identical data or the Topic's messages spread among defined partitions and each partition includes subset of data? 2. Whether adding broker to Kafka cluster is dynamic or do I need restart the existing broker in case of

How to discover broker Addresses for Producer

Hi I am implementing a producer. * ZkClient zkClient = new ZkClient(zookeeperUrl, 4, 6, new BytesPushThroughSerializer());* * List brokerIdList = zkClient.getChildren("/brokers/ids");* I am able to get Ids of the brokers. But I couldn't figure out how to get the urls

How to set Group Id in SimpleConsumer

Hi, I just wonder how can I assign GroupId when I use SimpleConsumer ( https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example) for fetching data from Kafka. Is it the last parameter of SimpleConsumer (leadBroker, a_port, soTimeOut, buffer_size, clientName) ? (i.e. clientN

Replay messages

Hi, Is it possible to replay only specific message in Kafka? For example message offsets are 1,2,3,4,5,6 a) Is it possible to replay only message with offset 3? b) is it possible to replay messages from [1-4] and 6? Thanks, Sourav -- Sourav Chandra Senior Software Engineer · · · · · · · ·

Re: trunk unit test failure

The fix is committed on 0.8.1 branch and trunk now. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop *

Re: Reg Exception in Kafka

Can you make the connection from the controller host to the other broker? Also, what's the # open file handlers on each broker? Do you see any "too many open file handler" error in the broker log? Thanks, Jun On Wed, Feb 19, 2014 at 12:14 AM, Balasubramanian Jayaraman (Contingent) < balasubrama

Re: Partitions and Scale out

1. Messages are spread among partitions in a topic. 2. Just added https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-CanIaddnewbrokersdynamicallytoacluster ? Thanks, Jun On Wed, Feb 19, 2014 at 4:10 AM, Erwin Karbasi wrote: > Hello Masters, > > I have 2 questions: > > 1. Whether all T

Re: How to discover broker Addresses for Producer

You can do a ZK get on /brokers/ids/broker_id to find out the broker host/port. You can also configure a vip for metadata.broker.list and then dynamically change the underlying hosts as brokers change. Also, you don't need to put all brokers in metadata.broker.list. Thanks, Jun On Wed, Feb 19,

Re: Partitions and Scale out

Thanks a lot for fast turnaround response. One more clarification, If I don't specify the partition number in producer so the message is inserted randomly in partitions. So if i have 2 partitions the messages are split out randomly in the topic's partitions and each topic will has different insert

RE: New Consumer API discussion

I am not clear on why the consumer stream should be positionable, especially if it is limited to the in-memory fetched messages. Could someone explain to me, please? I really like the idea of committing the offset specifically on those partitions with changed read offsets, only. 2 items I w

Re: How to set Group Id in SimpleConsumer

Group management like load balancing only exists in high level consumers, SimpleConsumer do not have the group id settings since it does not have group management feature. Guozhang On Wed, Feb 19, 2014 at 5:49 AM, shahab wrote: > Hi, > > I just wonder how can I assign GroupId when I use Simple

Re: some brokers cannot register themselves with zookeeper

Hi Clark, I think 992 is in 0.8 final release, but not in 0.8-beta. Guozhang On Fri, Feb 14, 2014 at 4:47 PM, Clark Breyman wrote: > Guozhang, > > All the patches for KAFKA-992 look like they were committed in August, > which was before 0.8 was shipped. Should we really be seeing this on 0.8?

Re: Replay messages

Hi Sourav, yes you can do that. Take a look at the Simple Consumer API https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Exampleand the SimpleConsumerShell https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/tools/SimpleConsumerShell.scalauses it as an implement

Re: Partitions and Scale out

Yes. Thanks, Jun On Wed, Feb 19, 2014 at 8:34 AM, Erwin Karbasi wrote: > Thanks a lot for fast turnaround response. > > One more clarification, If I don't specify the partition number in producer > so the message is inserted randomly in partitions. > So if i have 2 partitions the messages are