Re: Can ActiveMQ take advantage of Oracle transparent application failover?
This question was posted May 16, 2011; 11:18am — by johanhonline johanh but I cannot locate any follow-up or resolution. Currently (04/25/2017) we have the same issue and I need to understand how to integrate Active MQ with Oracle 12.1.0.2 TAF (Transparent Application Failover) so that bouncing one node of the Oracle RAC doesn't break Active MQ as described in the original post below. I have read and understand the Oracle TAF documentation and combed through Active MQ lists. Just need some input on how to get Active MQ to integrate with Oracle TAF. Any pointers to manuals/documentation or resources describing tests (successful or not) to get Active MQ to work with Oracle TAF are appreciated. Thanks -- bsm...@skylinenet.net johanh wrote > Hi list, > > We are currently running a master-slave cluster of ActiveMQ nodes, with a > JDBC persistence adapter backed by an Oracle 11g RAC database. This is > working fine from an ActiveMQ perspective; when the master goes down, a > slave automatically becomes the new master. However, we are having issues > when one of the database nodes goes down, and I am looking for ways to > have the ActiveMQ nodes handle this event as seamlessly as possible: > > When one of the database nodes goes down, the master ActiveMQ node loses > its database lock, with the database returning an "ORA-25402: transaction > must roll back error". Per our DBA, this is normal behavior for Oracle > RAC, and is the Oracle OCI JDBC driver's cue to retry the connect. > However, this error causes the master ActiveMQ node to shut down entirely: > > After this, the earlier master node is completely shut down (the process > is no longer running), and none of the earlier slave nodes have taken over > as master. Restarting the master node ActiveMQ service makes it reacquire > the lock and become master again. Comparing these observations to what we > see in the database, it seems that the OCI driver is attempting to fail > over the connection to another database node (we can see some connections > from it failing over), but apparently that process gets interrupted by the > ActiveMQ master node shutting down per the above. Is there some way in > ActiveMQ to have the master node either attempt to reacquire the lock > itself, or wait for some time before shutting down, or is there some other > way in which database transparent application failover has been > implemented in ActiveMQ? > > Any advice appreciated! > > Regards, > Johan H -- View this message in context: http://activemq.2283324.n4.nabble.com/Can-ActiveMQ-take-advantage-of-Oracle-transparent-application-failover-tp3526485p4725226.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Can ActiveMQ take advantage of Oracle transparent application failover?
Tim, Thanks for the detailed reply. I will share with our app design/development team for more in-depth discussion and post back with status on any workarounds or coding we come up with. Also, may consider entering a feature request as you suggested. Agree it may be a bigger update than team wants to tackle but if nothing else it may stir some more discussion and brainstorming. I've got to believe someone somewhere has had the same architecture with similar problem as follows: Oracle RAC, Active MQ slave/master - take down one node of the RAC (for maintenance or unexpected problem) and Active MQ survives on the remaining node. Just to clarify, I believe the slave broker didn't acquire the lock because it appears both master and slave were still pointed to the Oracle RAC node that was taken down for maintenance (node 2 was left up and we expected Active MQ to connect to the DB though that node. We’re using an Oracle SCAN Listener - Single Client Access Name). After we did maintenance, and rebooted node 1 of the RAC, the Active MQ was back and live (on that node). So... may be able to do some scripting and redirecting from the Oracle side when a RAC nodes goes down... we'll see. Again, thanks. You or anyone else with more suggestions or experiences to share, be glad to hear and keep the thread alive. Thanks, -- bsm...@skylinenet.net -- View this message in context: http://activemq.2283324.n4.nabble.com/Can-ActiveMQ-take-advantage-of-Oracle-transparent-application-failover-tp3526485p4725240.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Can ActiveMQ take advantage of Oracle transparent application failover?
Thanks Steve, good point. In this standard Oracle RAC configuration the SCAN listener effectively plays the role of load balancer. Active MQ connects to the database through port 1610 and then is directed to node a or node b on port 1521. I’ve found an Oracle DocID: How To Configure Server Side Transparent Application Failover (Doc ID 460982.1) with which I going to experiment. This as opposed client-side where somehow modifying Active MQ to include the OCI (Oracle Call Interface) code to perform the node switch. It looks like it will depend on how the Active MQ is querying the ACTIVEMQ_LOCK table and how often if Oracle will seamlessly redirect to the surviving node. Thanks for comments and suggestions, please keep them coming. I’ll post progress. Thanks, -- bsm...@skylinenet.net -- View this message in context: http://activemq.2283324.n4.nabble.com/Can-ActiveMQ-take-advantage-of-Oracle-transparent-application-failover-tp3526485p4725245.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
