Theo,
Wondering if changing those settings as suggested worked for you?
Thanks,
Neha
On Wed, Sep 3, 2014 at 12:51 AM, Theo Hultberg wrote:
> that makes sense. if the size of each fetch is small then compression won't
> do much, and that could very well explain the increase in bandwidth.
>
> we
that makes sense. if the size of each fetch is small then compression won't
do much, and that could very well explain the increase in bandwidth.
we will try to change these settings and see what happens.
thanks a lot for your help.
T#
On Tue, Sep 2, 2014 at 10:44 PM, Guozhang Wang wrote:
> H
Hi Theo,
You can try to set replica.fetch.min.bytes to some large number (default to
1) and increase replica.fetch.wait.max.ms (default to 500) and see if that
helps. In general, with 4 fetchers and min.bytes to 1 the replicas would
effectively exchange many small packets over the wire.
Guozhang
Hi Guozhang,
We're using the default on all of those, except num.replica.fetchers which
is set to 4.
T#
On Mon, Sep 1, 2014 at 9:41 PM, Guozhang Wang wrote:
> Hello Theo,
>
> What are the values for your "replica.fetch.max.bytes",
> "replica.fetch.min.bytes", "replica.fetch.wait.max.ms" and
>
Hello Theo,
What are the values for your "replica.fetch.max.bytes",
"replica.fetch.min.bytes", "replica.fetch.wait.max.ms" and
"num.replica.fetchers" configs?
Guozhang
On Mon, Sep 1, 2014 at 2:52 AM, Theo Hultberg wrote:
> Hi,
>
> We're evaluating Kafka, and have a problem with it using more
Hi,
We're evaluating Kafka, and have a problem with it using more bandwidth
than we can explain. From what we can tell the replication uses at least
twice the bandwidth it should.
We have four producer nodes and three broker nodes. We have enabled 3x
replication, so each node will get a copy of a
