g in the consumer? If so, you can use the
> maxLag/minFetch JMX described in
> http://kafka.apache.org/documentation.html#monitoring
>
> Thanks,
>
> Jun
>
>
> On Thu, Aug 22, 2013 at 7:41 AM, Pablo Barrera González <
> pablo.barr...@gmail.com> wrote:
>
>
Hello
I am trying to monitor our kafka cluster. For that I use a zookeeper client
that reads the information of all the consumer and a jmx agent that
connects to the brokers. I try to look for discrepancies between the data
being produced and the data being consumed.
I dislike the idea of using t
t;
> Thanks,
> Neha
>
> On Fri, Apr 12, 2013 at 10:56 AM, Pablo Barrera González
> wrote:
>> Hi
>>
>> Is it possible to enable compression between the broker and the consumer?
>>
>> We are thinking in develop this feature in kafka 0.7 but first I would
&g
Hi
Is it possible to enable compression between the broker and the consumer?
We are thinking in develop this feature in kafka 0.7 but first I would
like to check if there is something out there.
Our escenario is like this:
- the producer is a CPU bounded machine, so we want to keep the CPU
cons
ve buffer size and determine whether the setting in the OS networking
> layer also needs to be adjusted. Likewise, you will need to use higher
> connection/session timeouts for zookeeper and set your offset commit
> intervals to be fairly large.
>
> Thanks,
>
> Joel
>
&
Hello
In my enterprise we are deploying an cross-site installation of Kakfa. One
of the Kafka cluster is located in USA and one consumer is in Europe. Does
anybody have experience in such an environment? Any comments on the
configuration and best practices?
Thanks in advance
Pablo
Jira ticket https://issues.apache.org/jira/browse/KAFKA-687
2013/1/7 Pablo Barrera González
> Thank you Jun and Neha
>
> I was trying to avoid adding more partitions. I have enough partitions if
> you count all partitions in all topics. I understand the problem with
> differen
et. Part of the reason is that we need to figure out how to balance
> > the data across topics since they can be of different sizes. We can look
> > into that post 0.8.
> >
> > For now, the solution is to increase the number of partitions on the
> > broker.
> >
>
Hello
We are starting to use Kafka in production but we found an unexpected (at
least for me) behavior with the use of partitions. We have a bunch of
topics with a few partitions each. We try to consume all data from several
consumers (just one consumer group).
The problem is in the rebalance ste
