Hi Girish,
Thank you for raising concerns about the proposed feature. I would
like to address the points you mentioned in your email.
1. I understand that one of your concerns is whether the proposed
regex-based deletion feature would provide significant advantages over
using a simple one-liner script to call the delete topic command for
each topic. As Yubiao pointed out, using scripts to delete topics one
by one can lead to increased network overhead and slow performance,
particularly when dealing with a large number of topics. Implementing
regex support for delete operations would provide a more efficient and
convenient way to manage resources in Pulsar.
2. In addition to the benefits for testing purposes, we have
communicated with business users of Pulsar and found that the proposed
regex-based deletion feature can be helpful in production environments
as well. For instance, it can be used to efficiently clean up
subscriptions associated with deprecated services, ensuring better
resource management and reducing clutter in the system.
3. As I suggested earlier, we can introduce a new option flag (e.g.,
`--regex` or `--pattern`) to the existing `pulsar-admin topics delete`
command to prevent breaking changes for users who have already used
the command in their scripts. This would ensure backward compatibility
while providing the new functionality for those who want to use regex
for deletion.
I hope this clears up any confusion and addresses your concerns.
Please let me know if you have any further questions or suggestions.
Best regards,
Xiangying Meng
On Mon, Apr 24, 2023 at 6:23 PM Girish Sharma <scrapmachi...@gmail.com> wrote:
>
> Hello Yubiao,
> As per my understanding, this feature suggestion is intended to delete the
> topics from all replicated clusters under the namespace. Thus, the example
> you are providing may not be a good fit for this?
>
> Xiangying, please clarify if my understanding is incorrect.
>
> On Mon, Apr 24, 2023 at 3:24 PM Yubiao Feng
> <yubiao.f...@streamnative.io.invalid> wrote:
>
> > Hi Girish Sharma
> >
> > > What additional advantage would one get by using that approach
> > > rather than simply using a one liner script to just call delete
> > > topic for each of those topics if the list of topics is known.
> >
> > If users enabled `Geo-Replication` on a namespace in mistake(expected
> > only to enable one topic),
> > it is possible to create many topics on the remote cluster in one second.
> >
> > Not long ago, 10,000 topics were created per second because of this
> > mistake. It took us a long time to
> > remove these topics. We delete these topics in this way:
> > ```
> > cat topics_name_file | awk '{system("bin/pulsar-admin topics delete "$0)}'
> > )
> > ```
> > It deletes topics one by one.
> >
> > We conclude later that stress test tools such as `Jmeter` or `ab` should be
> > used to delete so many topics.
> >
> > If Pulsar could provide these APIs, it would be better.
> >
> > Thanks
> > Yubiao Feng
> >
> >
> >
> >
> > On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <scrapmachi...@gmail.com>
> > wrote:
> >
> > > Hello Yubiao,
> > >
> > > What additional advantage would one get by using that approach rather
> > than
> > > simply using a one liner script to just call delete topic for each of
> > those
> > > topics if the list of topics is known.
> > >
> > > Regards
> > >
> > > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > > <yubiao.f...@streamnative.io.invalid> wrote:
> > >
> > > > In addition to these two, It is recommended to add a method to batch
> > > delete
> > > > topics, such as this:
> > > >
> > > > ```
> > > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > > >
> > > > or
> > > >
> > > > pulsar-admin topics delete-all-topic <a file contains a topic_name
> > lists>
> > > > ```
> > > >
> > > > Thanks
> > > > Yubiao Feng
> > > >
> > > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xiangy...@apache.org>
> > > > wrote:
> > > >
> > > > > Dear Apache Pulsar Community,
> > > > >
> > > > > I hope this email finds you well.I am writing to suggest a potential
> > > > > improvement to the Pulsar-admin tool,
> > > > > which I believe could simplify the process of cleaning up tenants
> > and
> > > > > namespaces in Apache Pulsar.
> > > > >
> > > > > Currently, cleaning up all the namespaces and topics within a tenant
> > or
> > > > > cleaning up all the topics within a namespace requires several manual
> > > > > steps,
> > > > > such as listing the namespaces, listing the topics, and then deleting
> > > > each
> > > > > topic individually.
> > > > > This process can be time-consuming and error-prone for users.
> > > > >
> > > > > To address this issue, I propose the addition of a "clear" parameter
> > to
> > > > the
> > > > > Pulsar-admin tool,
> > > > > which would automate the cleanup process for tenants and namespaces.
> > > > Here's
> > > > > a conceptual implementation:
> > > > >
> > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > ``` bash
> > > > > pulsar-admin tenants clear <tenant-name>
> > > > > ```
> > > > > 2. To clean up all topics within a namespace:
> > > > > ```bash
> > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > ```
> > > > >
> > > > > By implementing these new parameters, users would be able to perform
> > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > I believe this improvement would greatly enhance the user experience
> > > when
> > > > > working with Apache Pulsar.
> > > > >
> > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > feedback
> > > > > from the community.
> > > > > If everyone agrees, I can work on implementing this feature and
> > submit
> > > a
> > > > > pull request for review.
> > > > >
> > > > > Looking forward to hearing your thoughts on this.
> > > > >
> > > > > Best regards,
> > > > > Xiangying
> > > > >
> > > >
> > >
> > >
> > > --
> > > Girish Sharma
> > >
> >
>
>
> --
> Girish Sharma
