Thanks Neha,
Unfortunately, the maintenance overhead of 2 more clusters is not acceptable to
us.
Would you accept a pull request on mirror maker that would rename topics on the
fly?
For example by accepting the parameter rename:
—rename src1/dest1,src2/dest2
or, extended with RE support:
—rename old_(.*)/new_\1
Kind regards,
Erik.
Op 20 okt. 2014, om 16:43 heeft Neha Narkhede <[email protected]> het
volgende geschreven:
> Another way to set up this kind of mirroring is by deploying 2 clusters in
> each DC - a local Kafka cluster and an aggregate Kafka cluster. The mirror
> maker copies data from both the DC's local clusters into the aggregate
> clusters. So if you want access to a topic with data from both DC's, you
> subscribe to the aggregate cluster.
>
> Thanks,
> Neha
>
> On Mon, Oct 20, 2014 at 7:07 AM, Erik van oosten <
> [email protected]> wrote:
>
>> Hi,
>>
>> We have 2 data centers that produce events. Each DC has to process events
>> from both DCs.
>>
>> I had the following in mind:
>>
>> DC 1 | DC 2
>> events | events
>> + + + | + + +
>> | | | | | | |
>> v v v | v v v
>> +----------------+ | +----------------+
>> | Receiver topic | | | Receiver topic |
>> +----------------+ +----------------+
>> | | mirroring | |
>> | | +------------------+ |
>> | | | |
>> | +--------------------+ |
>> v v v v
>> +----------------+ | +----------------+
>> | Consumer topic | | | Consumer topic |
>> +----------------+ | +----------------+
>> + + + | + + +
>> | | | | | | |
>> v v v | v v v
>> consumers | consumers
>>
>>
>> As each DC has a single Kafka cluster, on each DC the receiver topic and
>> consumer topic needs to be on the same cluster.
>> Unfortunately, mirror maker does not seem to support mirroring to a topic
>> with another name.
>>
>> Is there another tool we could use?
>> Or, is there another approach for producing and consuming from 2 DCs?
>>
>> Kind regards,
>> Erik.
>>
>> —
>> Erik van Oosten
>> http://www.day-to-day-stuff.blogspot.nl/
>>
>>
