Makes sense. I did not fully appreciate the importance of the
"filtered" boolean on the CommandGetTopicsOfNamespaceResponse message.
I agree that it could be valuable to detect "simple" regexes and apply
those filters server side. That could also be an optimization to add
later, if it proves to be
On Thu, Mar 10, 2022 at 12:26 PM Michael Marshall wrote:
> > I believe we can address the risk by providing a switch to enable/disable
> > this feature.
>
> Would it be worth putting server side filtering behind a flag in the
> FeatureFlags protobuf message so that clients can still benefit from
Thank you for your responses, Matteo and Andras. This PIP is a good
improvement on the current design for regex consumers. I agree that
there isn't an easy way to distribute new topic notifications via a
topic, at this time.
> This proposal doesn't really change the existing behavior of how that
>
--
Matteo Merli
On Tue, Mar 8, 2022 at 9:36 PM Michael Marshall wrote:
>
> My primary concern is that our event driven messaging platform relies
> on polling for distribution of system events, like newly created
> topics. I would love to find a purely event driven solution.
As discussed above,
My primary concern is that our event driven messaging platform relies
on polling for distribution of system events, like newly created
topics. I would love to find a purely event driven solution.
I have some additional questions before I reply to the above feedback.
How will this feature affect P
Hi Michael,
Thanks for having had a look at the proposal.
I share Matteo's concerns about the system topic.
> One of my concerns is that this feature only solves the problem of
> topic discovery for regex consumers. Topic discovery is a generic
> problem in Pulsar, and any solution we implement
On Wed, Mar 2, 2022 at 2:15 PM Michael Marshall wrote:
> > A new class, org.apache.pulsar.TopicsListService will keep track
> > of watchers and will listen to changes in the metadata.
>
> I think we should avoid creating a new service to distribute
> notifications to consumers. Instead, we should
Hi Andras Beni,
Thanks for your detailed proposal. I agree that the regex consumer is
a good candidate for optimization. Were there any alternative
solutions you considered when looking to optimize the regex consumer,
and if so, will you describe them?
One of my concerns is that this feature only
Hi everyone,
I've just created a proposal for improving the performance of regex
subscriptions.
It's available at https://github.com/apache/pulsar/issues/14505 and is
copied below.
Let me know what you think.
Thanks,
Andras
Motivation
Pulsar allows consumers to subscribe to multiple topics by a
