Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread Tim Bain
Also, let's establish for sure whether the problem is that the Camel route is occasionally failing to publish to the topic, or that messages make it to the topic but are lost before consumption. Add a log line the prints the current ID as the very last step in your Camel route; if there are gaps,

Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread Tim Bain
You can quickly test that by inspecting the subscription via JMX on the broker; it'll have a flag that tells you whether it's durable. If not, submit a bug in JIRA. On Mar 1, 2016 8:08 PM, "mtod" wrote: > Sorry yes I did read his post. > > I was already using a durable subscription. > > C# code.

Re: 5.13.1 message blocked

2016-03-01 Thread yang.yang.zz
The question is "what is 138.42.247.41" is waiting for? Even the response is never coming back, why it's waiting forever and block other transport threads. Is there a way to timeout this forever wait? -- View this message in context: http://activemq.2283324.n4.nabble.com/5-13-1-message-blocked-

Re: 5.13.1 message blocked

2016-03-01 Thread yang.yang.zz
More findings in the consumer broker0. Why some connections a blocked. Here just give an example of a good connection for comparison

Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread mtod
Sorry yes I did read his post. I was already using a durable subscription. C# code. public SimpleTopicSubscriber(string topicName, string brokerUri, string clientId, string consumerId, string filter, string userID, string password, string message) { try {

Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread Tim Bain
Did you read Art's post where he suggested that your problem is that you're breaking the network connection between brokers with non-durable topic subscriptions, which is expected to lose messages? To test the theory, temporarily change your consumers to use durable subscriptions and see if the me

Active MQ performance tuning

2016-03-01 Thread vishva
Hi, I have configured activemq on one of my servers. Whenever my Apache- Tomee server 1.7.2 receives a request, i make a call to activemq to poll for an object. I am getting around 500 calls per second. When I start my server, I start getting following exception in around 1 hr. javax.jms.JMSExcept

Re: 5.13.1 message blocked

2016-03-01 Thread yang.yang.zz
Thanks Tim for responding.. I just run the same test again and happen to have some connected brokers and disconnected brokers after the test. By looking at the thread comparison, I see the difference between connected broker & disconnected (broken) brokers is

Re: Failover very slow with kahadb while restart of master is fast

2016-03-01 Thread artnaseef
Does the shutdown of the broker complete cleanly in both cases? A slow shutdown will lead to the shutdown script using kill -9, which eliminates any possibility of the process continuing to ensure a clean shutdown. In that case, the contents of the file may be left in a dirty state, which may lea

Re: question for users of NFS master/slave setups

2016-03-01 Thread artnaseef
So 15 seconds sounds really low, although I'm not sure of all the various timeout settings in NFS. Specifically here, the timeout of concern is the release of a lock held by a client. The higher the timeout, the less likelihood of two clients obtaining the same lock, but the slower failover becom

Re: 5.13.1 message blocked

2016-03-01 Thread yang.yang.zz
Hi Tim I collected 3 broker logs during the network blocking events. Which are respectively: consumer broker log (broker0), and a good producer broker (still connected after the network resumes) and a bad producer broker (disconnected after network resumes) consumer_broker.txt

Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread mtod
To expand on model a bit : The model is currently designed to support a Pub/Sub model. We have several applications that publish documents and events to ActiveMQ. Each application is assigned a queue each message that is submitted to the queue is placed in an XML wrapper (see below) using a Camel

Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread mtod
Thanks for all the help. I removed the chaos monkey and started a new run. I don't think I will see any errors this model has been running in my production for 2 years. The only difference is that this is using a newer version of ActiveMQ and LevelDB on a Linux cluster. Granted I have not had to

Re: activemq consumer does not return data even when queue not empty

2016-03-01 Thread vishva
Thanks artnaseef, for pointing me in the right direction.The issue was mainly because it was running under JTA transaction, and hence the consumers were not releasing the connection. So after around 50-100 polls, all the connections were exhausted. I was able to fix it by making TransactionSuppor

Re: question for users of NFS master/slave setups

2016-03-01 Thread Christian Schneider
I have seen a similar scenario that I described in a mail on the user list http://activemq.2283324.n4.nabble.com/Failover-very-slow-with-kahadb-while-restart-of-master-is-fast-tp4707500.html Do you have any idea why the failover is slower than a simple restart of the master? Christian On 28.0

Re: Integrate ActiveMQ 5.13.1 into WildFly 10.0.0

2016-03-01 Thread THMayr
One update: You don't have to reference the ActiveMQ resource adapter from the EJB3 subsystem in the WildFly configuration file. This defines only the default messaging system used by WildFly, when no explicit message broker is defined in the deployment descriptor of the EJB application. Instead

Connections tab in admin console

2016-03-01 Thread gyani
I have configured activemq and client on ssl , but in the the connections tab remote address is shown as tcp://127.0.0.1:58951 I expect it to be ssl. Or is there any other way to verify that the protocol is indeed ssl. -- View this message in context: http://activemq.2283324.n4.nabble.com/Conn

Embedded Broker Monitoring

2016-03-01 Thread Pat0675
Hi, in my application (its a standalone jar) I also start an embedded broker. I can monitor this via the jmx console. So my question is now, is it possible to launch anything programmatically to get access via the jolokia or the hawtio interface? And can I monitor more than one broker per JVM? T

Re: question for users of NFS master/slave setups

2016-03-01 Thread James A. Robinson
On Tue, Mar 1, 2016 at 7:41 AM, Tim Bain wrote: > Another possibility: the paths that each broker uses to reach the lock file > don't resolve to the same file in NFS. > In my case they resolve to the same server IP and export path.​

Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread Tim Bain
Excellent catch, Art. On Mar 1, 2016 8:38 AM, "artnaseef" wrote: > I missed something critical here - Topics. > > Topic flow across a network of brokers with non-durable subscriptions only > is not reliable. Brokers subscribe to one-another in the same manner that > end-clients subscribe to a br

Re: 5.13.1 message blocked

2016-03-01 Thread Tim Bain
Tell us more about "they're no longer communicating to each other." In a JMX viewer, for a particular queue (pick one, doesn't matter which) on each broker, which consumers are present on that queue before and after the restart? Also, what's in the logs of the brokers immediately before, during,

Re: question for users of NFS master/slave setups

2016-03-01 Thread Tim Bain
Another possibility: the paths that each broker uses to reach the lock file don't resolve to the same file in NFS. On Mar 1, 2016 8:29 AM, "artnaseef" wrote: > So something is very wrong then. NFS should *not* allow two NFS clients to > obtain the same lock. > > Three possible explanations come

RE: Daily Message Count - HELP

2016-03-01 Thread artnaseef
Clarifying Virtual Topics... When using Virtual Topics, there are both Topics and Queues involved. The Topic name starts with "VirtualTopic." (this can be changed via configuration). The Queue names start with "Consumer." The easiest way to visualize Virtual Topics is to think of an interceptor

Re: question for users of NFS master/slave setups

2016-03-01 Thread James A. Robinson
On Tue, Mar 1, 2016 at 7:02 AM, artnaseef wrote: > So something is very wrong then. NFS should *not* allow two NFS clients to > obtain the same lock. > > Three possible explanations come to mind: > > * The lock file is getting incorrectly removed (I've never seen ActiveMQ > cause this) > * There

Re: ActiveMQ not starting

2016-03-01 Thread Tim Bain
Look in the start script to determine how that path is built. On Mar 1, 2016 8:19 AM, "OrangeJuice" wrote: > I've just checked the paths. The ../lib/ one indeed doesn't exist, as that > one would be outside the folder where I extracted ActiveMQ to. > I've tried copying the lib folder to that path

Re: activemq consumer does not return data even when queue not empty

2016-03-01 Thread artnaseef
Thanks for posting your findings. That's an interesting fix given the fact that the JMS spec states that a JMS connection created within a JTA transaction participates in that transaction automatically (although I'm sure it's fine for a provider to allow a means to pull it out of the transaction).

Re: Help with a Failover testing that shows missing messages

2016-03-01 Thread artnaseef
I missed something critical here - Topics. Topic flow across a network of brokers with non-durable subscriptions only is not reliable. Brokers subscribe to one-another in the same manner that end-clients subscribe to a broker. So, for topic subscriptions, that means that while the bridge between

Re: question for users of NFS master/slave setups

2016-03-01 Thread artnaseef
As far as a broker properly cleaning up when active and losing the lock -- first off, that's a very rare scenario. With that said, there's no way to guarantee a completely clean hand-off at that point. The cause of such a scenario will be a drop in network communication between that broker and th

Re: question for users of NFS master/slave setups

2016-03-01 Thread artnaseef
So something is very wrong then. NFS should *not* allow two NFS clients to obtain the same lock. Three possible explanations come to mind: * The lock file is getting incorrectly removed (I've never seen ActiveMQ cause this) * There is a flaw in the NFS locking implementation itself * The NFSv4 t

Re: ActiveMQ not starting

2016-03-01 Thread OrangeJuice
I've just checked the paths. The ../lib/ one indeed doesn't exist, as that one would be outside the folder where I extracted ActiveMQ to. I've tried copying the lib folder to that path, but the error message persisted. Just downloaded the same archive to a linux mint version. Works flawlessly ther

Re: ActiveMQ not starting

2016-03-01 Thread Tim Bain
Asking the obvious question: do all of the paths in that command exist? If not, which ones don't? In particular, does /home/nico/activemq/apache-activemq-5.13.1//../lib/ exist? (Check them all, but that one in particular looks fishy.) On Mar 1, 2016 1:26 AM, "OrangeJuice" wrote: > I've tried s

Re: question for users of NFS master/slave setups

2016-03-01 Thread Tim Bain
That's better than the impression I'd gotten last time I investigated the question. Do you get more useful information at DEBUG? And do you get the same behavior if you wait to start 2c till 2a is fully up? On Mon, Feb 29, 2016 at 7:08 PM, artnaseef wrote: > Something sounds very wrong there.

Re: Recommendation for vendor independent strategy

2016-03-01 Thread Peter Hansson
Thanks to all who have replied. Appreciated. It kinda confirmed my suspicion about current situation. Anyways: JMS has been good for us. It has played its part in being a (rather simple) vendor agnostic facade to various MOMs. I think we'll skip the ActiveMQ case for now. Peter

Re: Production ConnectionFactory configuration with Spring

2016-03-01 Thread itelleria
>From my point of view, one difference between SingleConnectionFactory and PooledConnectionFactory is that SingleConnectionFactory reuse the same connection among all the listeners/consumers and PooledConnectionFactory shares a pooled of connections among the listeners/consumers. I think if I use

Re: ActiveMQ not starting

2016-03-01 Thread OrangeJuice
I've tried setting the evironment variable ACTIVEMQ_HOME to the folder where I've extracted ActiveMQ without the trailing slash. Now the double slashes are gone, but it still doesn't work. > $ ./activemq console INFO: Loading '/etc/default/activemq' INFO: Using java '/usr/bin/java' INFO: Startin

Re: ActiveMQ not starting

2016-03-01 Thread OrangeJuice
No modifications to the script, just downloaded the file, extracted it and tried running ActiveMQ. I've uploaded the entire output to pastebin, as it's rather long. http://pastebin.com/Chzg7gb1 -- View this message in context: http://activemq.2283324.n4.nabbl