Hi,
We have an interesting problem to solve due to a very large traffic volumes
on particular topics. In our initial system configuration we had only one
partition per topic, and in in a couple of topics we have built up huge
backlogs of several million messages that our consumers are slowly
proce
This will force a rewrite of those monitoring tools and UI tools that read
offsets from Zookeeper in order directly to get lag information for
reporting on consumer clients. It seems a good thing to know this is coming
down the pipe.
Dennis
On Wed, Jun 4, 2014 at 6:50 PM, Neha Narkhede
wrote:
rted list
> of
> > consumer instances. This makes the rebalancing algorithm deterministic.
> As
> > soon as you bring up the 2nd consumer instance, if its position in the
> > sorted list is before the position of the 1st consumer client, it will
> end
> > up owning the p
ossible to
> predict which consumer will claim the ownership of the partition.
>
> Guozhang
>
>
> On Thu, Aug 22, 2013 at 8:53 AM, Dennis Haller >wrote:
>
> > I have a situation where two high level consumers are being created to
> > consume a single topic. There
I have a situation where two high level consumers are being created to
consume a single topic. There is only one partition for the topic, so I
understand that only one Consumer will end up owning the topic and
receiving messages. The two consumers are created from two servers in a
redundant master
cause the broker closed the socket. Any
> error/exception on the broker side around the same time?
>
> Thanks,
>
> Jun
>
>
> On Mon, Jul 8, 2013 at 10:25 AM, Dennis Haller >wrote:
>
> > I have a 4-broker Kafka system running in Amazon EC2, and we are usi
I have a 4-broker Kafka system running in Amazon EC2, and we are using
Kafka 0.8 beta1. Most of the standard default configurations remain
unchanged. Running the kafka tool ConsumerOffsetChecker is causing socket
errors to occur. Some of these socket reset errors are also in the kafka
server log.
The version of zookeeper embedded with Kafka 0.8 is still v. 3.3.3.
Regards
Dennis
On Wed, Apr 24, 2013 at 10:41 AM, Yu, Libo wrote:
> According to what I tried, kafka 0.8 works with zookeepr 3.4.3.
>
>
> Regards,
>
> Libo
>
>
Hi,
When the zk.connect property of the Kafka server is specified with a comma
separated host:port list, such as "127.0.0.1:3000,127.0.0.1:3001,
127.0.0.1:3002"., does that mean that every ZK connection attempt goes to
the first listed server if it exists? And does that represent a risk of
driving
I'm not sure if this relates directly to your problem, but you are using
non-standard topic names because of the @ character. Topic names should
only include alphanumeric plus hyphen and underscore. Have you checked your
logs for any errors because of processing these topic names?
Regards,
Dennis
Hi,
We found the same applies for topic names.
Dennis
On Fri, Mar 29, 2013 at 7:57 AM, Jun Rao wrote:
> Chris,
>
> Client id is used for registering jmx beans for monitoring. Because of the
> restrictions in bean names, we limit the client id to be only alpha-numeric
> plus "-" and "_".
>
>
What exactly is a metric reporter - something in log4j?
Thanks
Dennis
On Wed, May 8, 2013 at 8:20 AM, Jun Rao wrote:
> 0.8 JMX is different from 0.7. In 0.8, all jmx beans are exposed through
> metrics. One can attach a metric reporter for monitoring.
>
> Thanks,
>
> Jun
>
>
> On Tue, May 7,
12 matches
Mail list logo
