Hi Jun,
Thanks for the quick response.
(I just discovered that you had answered a similar question on the dev list
-
http://mail-archives.apache.org/mod_mbox/kafka-dev/201304.mbox/%3CCAFbh0Q1XSjG2B9iO%2BpDeM8p9RsDDqKA9E8LY5ScXzAncaQ_mVg%40mail.gmail.com%3E
)
I set the following properties on the
In 0.7, each producer reconnects to the broker periodically. Messages sent
in different socket connections could arrive out of order. Take a look at
reconnect.interval in http://kafka.apache.org/07/configuration.html. If you
don't use VIP for the producers, you can actually set reconnect.interval t
Hi,
I am using Kafka 0.7.1, and using SyncProducer and SimpleConsumer with a
single broker service process.
I am occasionally seeing messages (from a *single* partition) being
processed out of order to what I expect and I am trying to find where the
problem lies. The problem may well be in my co
Right. Zookeeper has added the feature to allow dynamically reconfiguring
zookeeper clusters, but it is part of zookeeper 3.5.0.
Thanks,
Neha
On Tue, May 21, 2013 at 3:05 PM, Jason Rosenberg wrote:
> Thanks Neha,
>
> I see now that is true. It looks like it just needs to resolve all the
> hos
Thanks Neha,
I see now that is true. It looks like it just needs to resolve all the
hosts (but doesn't need zk to be available on each host). So
decommissioning a zknode (and removing from dns) is a bit more sensitive
than I thought. I can't think of any reason that zkClient should require
this
According to zookeeper code, when you try to create a client handle to
zookeeper, it resolves each of the hosts in the zookeeper connection
string. If any of these fail, it throws an exception. If you use a
zookeeper based producer, it tries to create a ZkClient handle which in
turn tries to create
Hi,
I am seeing an unexpected situation. My producers use a zkconnection
string to connect to kafka (this is still 0.7.2). If one of the zk hosts
is taken down and removed from dns, it causes an UnknownHostException, and
the producer can't initialize. I expect this is different than the less
se
It will be great to have a user group meeting around Hadoop Summit. I
prefer Thursday.
Thanks,
Neha
On Tue, May 21, 2013 at 10:54 AM, Vaibhav Puranik wrote:
> Any day will work for me. I am waiting for you to decide on time so that I
> can book my ticket accordingly!
>
> Regards,
> Vaibhav
>
>
Any day will work for me. I am waiting for you to decide on time so that I
can book my ticket accordingly!
Regards,
Vaibhav
On Tue, May 21, 2013 at 4:47 AM, Jonathan Hodges wrote:
> Tuesday or Thursday works the best for me as I plan to attend the Summit
> Party on Wednesday evening. Thanks!
Cool.
By the way, I do mean you should use 'atop'. That was not a typo on my part.
http://www.atoptool.nl/downloadatop.php
apt-get install atop
on Ubuntu systems.
Philip
On May 21, 2013, at 4:51 PM, Jason Weiss wrote:
> Philip,
>
> Thanks for the response. I used top yesterday and deter
Philip,
Thanks for the response. I used top yesterday and determined that part of
my problem was that the kafaka shell script is pre-configured to only use
512M of RAM, and thus it wasn't using memory efficiently. That has helped
out tremendously. Adding an echo at the start of the script that it
As a test, why not just use a disk with provisioned IOPs of 4000? Just as a
test - see if it improves.
Also, you have not supplied any metrics regarding the VM's performance. Is the
CPU busy? Is IO maxed out? Network? Disk? Use a tool like atop, and tell us
what you find.
Philip
On May 20, 2
Yes. This was added to get around the VIP hard killing connections that are
idle. If your producer doesn't use a VIP you can set this high enough so it
rarely reconnects.
Thanks,
Neha
On May 21, 2013 8:13 AM, "Jason Weiss" wrote:
> Ahh - I see, thank you.
>
> Would you describe this as a "tunabl
Ahh - I see, thank you.
Would you describe this as a "tunable" parameter to achieve higher
performance? For example, is it in a producer's interest to bump this
figure up to say 250,000 from 30,000?
Thanks again for the prompt response!
Jason
On 5/21/13 11:07 AM, "Jun Rao" wrote:
>In 0.7, ea
In 0.7, each producer reconnects to the broker periodically. Take a look at
reconnect.interval in http://kafka.apache.org/07/configuration.html
Thanks,
Jun
On Tue, May 21, 2013 at 7:35 AM, Jason Weiss wrote:
> I've made some good progress on tuning Kafka for our needs, reaching about
> 40,000
I've made some good progress on tuning Kafka for our needs, reaching about
40,000 events per second in our environment. One thing that I noticed in all of
the Kafka server logs is an inordinate amount (I think?) of Closing socket
connection INFO statements.
I realize they are INFO, so I assume
Tuesday or Thursday works the best for me as I plan to attend the Summit
Party on Wednesday evening. Thanks!
On Mon, May 20, 2013 at 10:05 PM, Jun Rao wrote:
> Yes, we can have a Kafka user group meeting then. We could do this in one
> of the evenings (Tue, Wed, or Thu). What will people prefe
17 matches
Mail list logo
