Re: Performance of high number of topics or selectors

2009-04-28 Thread Gary Tully
There are some hard limits in terms of fds and memory utilisation both for destinations and connections. The simplest approach to achieve scalability at these levels (100k topics) is to divide and conquer using a network of brokers and a random

Re: Performance of high number of topics or selectors

2009-04-28 Thread Aaron Mulder
Well, the send to everyone and manually filter turned out to be a really bad idea -- I guess the message gets copied too many times or whatever. It gets an OOM error with a 512 MB heap for 10,000 clients. So I'm still looking for a better way, whether it be general approach or tuning to handle hi

Performance of high number of topics or selectors

2009-04-27 Thread Aaron Mulder
Let's say I have 100,000 stock tickers (or whatever) I want to track, each with a separate client. There seem to be two approaches -- putting them all in one topic and having each of the clients use a selector (ticker equivalent in a header property), or putting messages for each header on it's ow