Hello, Per our deployment topology , we have a central cluster (5 brokers) and remote producers in various geographic locations(#100) (~30 hosts set up in each of geographic location). We will be publishing records at the rate of 10000 events/ sec(across all sites) to the cluster. Each event is 1K size and that corresponds to 10 MB/sec.
Per your prior experience with Kafka deployment, can you please provide any inputs for my questions below: 1. Do you see any issues with no.of remote publishers to the Kafka cluster? The network lag between the remote sites to the central cluster could be 70ms on average. 2. Do you see any limitations during kafka cluster expansions/upgrades with the multiple publishers in remote sites? 3. Do you recommend a proxy model here with rest interface deployed in proximity of the Kafka cluster doing the publishing? Any responses are highly appreciated. Thanks Srikala.
