Hey Otto, Yeah this isn't something we've really thought about. Presumably the implementation would be that the server accept connections on two interfaces. That is pretty easy. However the harder part is that I think this would require updating the metadata to advertise a different ip/host to other brokers versus to producers (right now there is just one for both). Or maybe there would be another way to do it?
-Jay On Wed, Mar 26, 2014 at 6:44 AM, Otto Mok <otto....@acuityads.com> wrote: > Hi Jay, > > We're pushing a lot of data from the producers (n) and have many consumers > (3n) reading them. > > We're configured to have replication factor of 3, so replication traffic > is about (2n). > > Currently all traffic was on a single NIC, so that's about (6n) total. > > Having the replication traffic on different IP/NIC would reduce the > bandwidth usage by 33%, down to (4n). > Or 50% more capacity for producers to push before hitting the NIC's cap (1 > Gbps) > > We're not quite at the cap yet, but would like to see if we can make use > of the second NIC to give us more room in the primary NIC. > > Thanks. > > Otto out! > > -----Original Message----- > From: Jay Kreps [mailto:jay.kr...@gmail.com] > Sent: March-25-14 6:22 PM > To: users@kafka.apache.org > Subject: Re: Separate broker replication traffic from producer/consumer > traffic > > No not at the moment. Are you seeing a problem that this would resolve? > > -Jay > > > On Tue, Mar 25, 2014 at 2:55 PM, Otto Mok <otto....@acuityads.com> wrote: > > > Hi all, > > > > Is there any way to configure the brokers such that producers & consumers > > are talking via IP1, while the brokers are replicating between themselves > > using IP2? > > > > I see there are broker settings for host.name and advertised.host.name, > > but it doesn't look like these settings does what I'm looking for. > > > > Any help or insights will be appreciated. > > > > Thanks. > > > > Otto out! > > > > >
