Re: Limitations on No.Of Producers and Consumers

2018-11-13 Thread Rajan Dhabalia
>> To clarify, region isn't really a pulsar concept here? You're just saying that these 3 cluster are geographically close? That's correct. 3 independent pulsar clusters but they are geographically close each other or present into the same data-center, can be peer to each-other. With this enhanceme

Re: Limitations on No.Of Producers and Consumers

2018-11-13 Thread Ivan Kelly
On Tue, Nov 13, 2018 at 8:23 PM, Rajan Dhabalia wrote: >> Is there any documentation on how to use it? > > I have documented peer-cluster setup at: > https://github.com/apache/pulsar/wiki/PIP-8:-Pulsar-beyond-1M-topics#how-to-setup-peer-clusters Great! We should turn this into a cookbook at some

Re: Limitations on No.Of Producers and Consumers

2018-11-13 Thread Rajan Dhabalia
> Is there any documentation on how to use it? I have documented peer-cluster setup at: https://github.com/apache/pulsar/wiki/PIP-8:-Pulsar-beyond-1M-topics#how-to-setup-peer-clusters Thanks, Rajan On Tue, Nov 13, 2018 at 4:22 AM Ivan Kelly wrote: > > After, PIP-8, It is now possible to run mu

Re: Limitations on No.Of Producers and Consumers

2018-11-13 Thread Ivan Kelly
> After, PIP-8, It is now possible to run multiple Pulsar clusters under one > service URL (going beyond the limits of a what a single ZK can support). I wasn't aware of that feature. Is there any documentation on how to use it? -Ivan

Re: Limitations on No.Of Producers and Consumers

2018-11-12 Thread Joe Francis
>>The real limit in pulsar is the amount of metadata that can be stored in zookeeper. After, PIP-8, It is now possible to run multiple Pulsar clusters under one service URL (going beyond the limits of a what a single ZK can support). Joe On Mon, Nov 12, 2018 at 7:36 AM, Dave Fisher wrote: > M

Re: Limitations on No.Of Producers and Consumers

2018-11-12 Thread Dave Fisher
Message was moderated onto the list, including the OP Sent from my iPhone > On Nov 12, 2018, at 2:28 AM, Ivan Kelly wrote: > > Hi Koushik, > >> I just got introduced to Apache pulsar and studying it to solve a problem at >> hand. >> As I understand that Pulsar supports 1M of topics, Is there

Re: Limitations on No.Of Producers and Consumers

2018-11-12 Thread Ivan Kelly
Hi Koushik, > I just got introduced to Apache pulsar and studying it to solve a problem at > hand. > As I understand that Pulsar supports 1M of topics, Is there any limitation on > no.of producers and subscribers that brokers can handle? I have the following > use cases. The limit of how many

Limitations on No.Of Producers and Consumers

2018-11-11 Thread koushik . infinite
Hi, I just got introduced to Apache pulsar and studying it to solve a problem at hand. As I understand that Pulsar supports 1M of topics, Is there any limitation on no.of producers and subscribers that brokers can handle? I have the following use cases. Usecase 1 : Few hundred producers pr