It's being lead by StreamNative, which provides a hosted Pulsar service. They haven't said it, but my gut's telling me that the end goal is to consolidate their serverless/hosted solution into a single cluster instead of needing to shard into multiple clusters.
- Nathan On Sun, Mar 16, 2025, at 19:31, Matt Pavlovich wrote: > I’d love to see the bill for storing metrics and logs for 1mm topics or > hundreds of millions of topics. > > That sounds more like a splash blog post vs a practical solution. > >> On Mar 8, 2025, at 1:06 PM, Nathan Clayton <apache....@nathanclayton.com> >> wrote: >> >> I'm not sure about Kafka, but Apache Pulsar is currently built to support up >> to 1mm topics. I know that there's active work on using Oxia instead of >> Zookeeper for the metadata store with the goal to support hundreds of >> millions of topics. >> >> That said, millions of topics in Artemis may be a bit of an anti-pattern. >> Depending on your use case, it may be better to use filter expressions on >> fewer queues [1]. >> >> Nathan >> >> [1] >> https://activemq.apache.org/components/artemis/documentation/latest/filter-expressions.html >> >> >> On Wed, Mar 5, 2025, at 12:45, William Crowell wrote: >>> I am wondering if Apache Kafka might be more feasible for something like >>> this. >>> >>> Regards, >>> >>> William Crowell >>> >>> From: Justin Bertram <jbert...@apache.org> >>> Date: Wednesday, March 5, 2025 at 3:26 PM >>> To: users@activemq.apache.org <users@activemq.apache.org> >>> Subject: Re: Maximum Amount of Topic/Queues Within Apache Artemis >>> That's it. >>> >>> >>> Justin >>> >>> On Wed, Mar 5, 2025 at 2:19 PM William Crowell >>> <wcrow...@perforce.com.invalid> wrote: >>> >>>> Justin, >>>> >>>> Never, I think I found it: >>>> >>>> “…the web console's behavior is configurable. Go to the "Preferences" >>>> (available from the menu in the top right) and click the "Jolokia" tab. >>>> Here you can turn off auto-refresh (i.e. "Update rate"). You can also >>>> decrease the amount of data fetched by lowering the "Max depth" and "Max >>>> collection size." >>>> >>>> Regards, >>>> >>>> William Crowell >>>> >>>> From: William Crowell <wcrow...@perforce.com.INVALID> >>>> Date: Wednesday, March 5, 2025 at 3:12 PM >>>> To: users@activemq.apache.org <users@activemq.apache.org> >>>> Subject: Re: Maximum Amount of Topic/Queues Within Apache Artemis >>>> Justin, >>>> >>>> Appreciate your reply. Your insight is greatly appreciated and invaluable. >>>> >>>> If you can provide more details on this, then I would appreciate it. I >>>> can search the archives for it as well. I am guessing there is some >>>> command line equivalent to represent the tree. >>>> >>>> What we have is several devices that have JMS clients that communicate >>>> with the broker on topics and durable queues. By “several devices” I mean >>>> maybe 100s or 1000s. >>>> >>>> Regards, >>>> >>>> William Crowell >>>> >>>> From: Justin Bertram <jbert...@apache.org> >>>> Date: Wednesday, March 5, 2025 at 2:57 PM >>>> To: users@activemq.apache.org <users@activemq.apache.org> >>>> Subject: Re: Maximum Amount of Topic/Queues Within Apache Artemis >>>> The broker itself doesn't impose any arbitrary limits on the number of >>>> addresses & queues. That said, there certainly are limits, the size of your >>>> heap probably being the most important. Every address and queue carry with >>>> it some memory overhead including the objects themselves but also related >>>> objects like JMX MBeans which enable management. >>>> >>>> The current management console will struggle by default with a huge number >>>> of addresses and/or queues due to the way the "tree" view works (i.e. >>>> refreshing the whole view on a regular basis). This is the main reason we >>>> removed this view in the new console [1]. That said, the console can be >>>> configured in such a way as to mitigate some of these problems. I can >>>> provide more details on that if necessary although it's been covered on >>>> this list a few times already. >>>> >>>> Regarding scalability...the broker was written with scalability in mind, >>>> but everything breaks down at some point. I'm interested to hear about your >>>> experiences if you go down this route especially regarding bottlenecks that >>>> you may find for your specific use-case. Keep us in the loop! >>>> >>>> >>>> Justin >>>> >>>> [1] >>>> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-5319&data=05%7C02%7CWCrowell%40perforce.com%7C320052ba6ccd49ef708808dd5c240792%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C638768032021655752%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=fiCybGio5R84gDL5NiT3uHHTrz%2FtKGU5k3AappaW%2FMc%3D&reserved=0<https://issues.apache.org/jira/browse/ARTEMIS-5319> >>>> < >>>> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-5319&data=05%7C02%7CWCrowell%40perforce.com%7C320052ba6ccd49ef708808dd5c240792%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C638768032021670531%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=x99X9Q4l3k5WDqM0eAAWMXIvYPR9ReXIlsBXyz3KBKo%3D&reserved=0<https://issues.apache.org/jira/browse/ARTEMIS-5319> >>>>> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-5319&data=05%7C02%7CWCrowell%40perforce.com%7C320052ba6ccd49ef708808dd5c240792%7C95b666d19a7549ab95a38969fbcdc08c%7C0%7C0%7C638768032021679546%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=dtoeTZ5yS%2BtWShgjBs6svkrqwbvLSy6Y4OPAjTTsUwQ%3D&reserved=0<https://issues.apache.org/jira/browse/ARTEMIS-5319>> >>>> >>>> On Wed, Mar 5, 2025 at 12:33 PM William Crowell >>>> <wcrow...@perforce.com.invalid> wrote: >>>> >>>>> Good afternoon, >>>>> >>>>> Are there any limitations or concerns with creating millions of topics >>>> and >>>>> queues within Apache Artemis that have low volumes of messages in each >>>>> topic and queue? I do not think there are as I believe Artemis should be >>>>> able to handle this use case. Important question is: Is it scalable? >>>>> >>>>> Regards and have a great day, >>>>> >>>>> William Crowell >>>>> >>>>> >>>>> This e-mail may contain information that is privileged or confidential. >>>> If >>>>> you are not the intended recipient, please delete the e-mail and any >>>>> attachments and notify us immediately. >>>>> >>>>> >>>> >>>> >>>> CAUTION: This email originated from outside of the organization. Do not >>>> click on links or open attachments unless you recognize the sender and know >>>> the content is safe. >>>> >>>> >>>> This e-mail may contain information that is privileged or confidential. If >>>> you are not the intended recipient, please delete the e-mail and any >>>> attachments and notify us immediately. >>>> >>>> >>>> This e-mail may contain information that is privileged or confidential. If >>>> you are not the intended recipient, please delete the e-mail and any >>>> attachments and notify us immediately. >>>> >>>> >>> >>> >>> CAUTION: This email originated from outside of the organization. Do not >>> click on links or open attachments unless you recognize the sender and >>> know the content is safe. >>> >>> >>> This e-mail may contain information that is privileged or confidential. >>> If you are not the intended recipient, please delete the e-mail and any >>> attachments and notify us immediately. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@activemq.apache.org >> For additional commands, e-mail: users-h...@activemq.apache.org >> For further information, visit: https://activemq.apache.org/contact >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@activemq.apache.org > For additional commands, e-mail: users-h...@activemq.apache.org > For further information, visit: https://activemq.apache.org/contact --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@activemq.apache.org For additional commands, e-mail: users-h...@activemq.apache.org For further information, visit: https://activemq.apache.org/contact
