Yes.. I think what the work around would probably do is revert to “softer” priority scheduling when it kicks in because the fall through message consumer with no selector will work.
I will still try to get a fix implemented of course but I have a release i need to get out the door :) On Fri, Apr 24, 2015 at 4:14 PM, Tim Bain <tb...@alumni.duke.edu> wrote: > It might be a good fail-safe, but hopefully you'll find something as you > step through with the debugger that will highlight what the problem > actually is (so you don't need to use the no-selector workaround). > Workarounds are great, but fixes are better. :) But it might be a way to > at least get production working again so you can solve this at a more > reasonable pace rather than spending 15-hour days on it. > > Tim > > On Fri, Apr 24, 2015 at 5:04 PM, Kevin Burton <bur...@spinn3r.com> wrote: > > > Sounds like a good idea. I just pushed with maxPageSize=5000 > > > > Interesting that my backlogged queues seems to be locked up at about 5000 > > messages. > > > > … I’m wonder if those are messages that aren’t matching selectors? > > > > I could have a plan B / fall through that accepts messages without a > > selector.. it would be a hack and would give me soft priority but this > > would get me closer to my goal. > > > > throttle-queue:discovery-task:pending:192.0.78.13 > > 5,004 4 > > throttle-queue:discovery-task:pending:192.0.78.12 > > 5,002 4 > > throttle-queue:discovery-task:pending:74.125.141.132 > > 5,000 0 > > throttle-queue:feed-task:pending:74.125.141.132 > > 4,997 0 > > throttle-queue:source-task:pending:69.171.237.20 > > 4,062 0 > > > > On Fri, Apr 24, 2015 at 3:40 PM, Tim Bain <tb...@alumni.duke.edu> wrote: > > > > > OK, so based on what you wrote, it sounds like the broker gets into a > > state > > > where message selectors don't work properly, since the same selector > > > matches all messages (except ones with a null priority; have you > > confirmed > > > that both artemis_priority <= 9 and artemis_priority = null match their > > > respective messages when you're in a good state, and that neither one > > gets > > > any messages when you're in a bad state?). > > > > > > I'd start the broker with the debug port enabled, pull up the > > > selector-evaluation code and set a breakpoint in it (I'd start with > > > SelectorWorker.run()), wait till you get into the bad state, and then > > > attach a debugger and step through the code to see why messages aren't > > > getting handed off to your consumer with the selector. > > > > > > Tim > > > > > > On Fri, Apr 24, 2015 at 4:27 PM, Kevin Burton <bur...@spinn3r.com> > > wrote: > > > > > > > > > > > > > > > http://imgur.com/a/2myja > > > > > > > > > > > > > > > What are the two screenshots; with and without the selector? If > > that's > > > > > right, then clearly zero messages are matching the selector (which > > > means > > > > > this isn't a delivery problem, it's a selector problem), so > hopefully > > > > > pulling that thread will lead to a solution. > > > > > > > > > > > > > > Actually, maybe it’s just a but related to message selectors? > > > > > > > > If I remove the selector, I have it print the messages it receives. I > > > then > > > > confirm that they are ALL have the right headers. So from a > > mathematical > > > > perspective, they should match. > > > > > > > > Here’s an example. > > > > > > > > QueueDumperTask: Got message: using config: Config{sync=true, > > > > selector='artemis_priority <= 9'} > > > > > > > > QueueDumperTask: > > > > ID:util0048.wdc.sl.spinn3r.com-38049-1429893012757-6:1:1:1:11179 > > > > > > > > QueueDumperTask: artemis.link= > > http://entertainment-2-all.blogspot.com > > > > > > > > QueueDumperTask: artemis.throttleKey=74.125.141.132 > > > > > > > > QueueDumperTask: artemis.pending.timestamp=2015-04-24T16:46:30Z > > > > > > > > QueueDumperTask: artemis.sequence=1429893990000012414 > > > > > > > > QueueDumperTask: > > > > > > artemis.pendingQueue=throttle-queue:discovery-task:pending:74.125.141.132 > > > > > > > > QueueDumperTask: artemis_priority=0 > > > > > > > > QueueDumperTask: artemis.throttleQueue=discovery-task > > > > > > > > QueueDumperTask: artemis.delay=0 > > > > > > > > QueueDumperTask: JMSXDeliveryCount=3 > > > > > > > > .. so it should match > > > > > > > > artemis_priority <= 9 > > > > > > > > … so I think there are two issues here. I’m aware of the improper > > > > scheduling issue do to maxPageSize… so I won’t get REAL priorities. > > I’ll > > > > get soft priorities. > > > > > > > > But that’s not the issue that I’m seeing.. What I’m seeing is that > the > > > > ENTIRE thing locks up. Because at least ONE of my selectors should > > work. > > > > > > > > > > > > > > If I’m already an in-memory store, is there any problem just > > setting > > > > > > maxPageSize to a HUGE number? > > > > > > > > > > > > > > > That's not code I've looked at, but I'd double-check the code to > make > > > > sure > > > > > it's not going to do anything crazy like pre-allocate a huge > > > > > array/ArrayList or anything like that... > > > > > > > > > > > > > Yes. That’s what I was worried about.. I might trace it or just try > it > > in > > > > production and see what memory usage looks like. > > > > > > > > -- > > > > > > > > Founder/CEO Spinn3r.com > > > > Location: *San Francisco, CA* > > > > blog: http://burtonator.wordpress.com > > > > … or check out my Google+ profile > > > > <https://plus.google.com/102718274791889610666/posts> > > > > <http://spinn3r.com> > > > > > > > > > > > > > > > -- > > > > Founder/CEO Spinn3r.com > > Location: *San Francisco, CA* > > blog: http://burtonator.wordpress.com > > … or check out my Google+ profile > > <https://plus.google.com/102718274791889610666/posts> > > <http://spinn3r.com> > > > -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts> <http://spinn3r.com>
