Hi, we get this error after activemq has been running for a while...and then it crashes all of a sudden. We think it might have something to do with what's been written in the DB. Over the last few days, it's been happening frequently - we'd need to drop the activemq DB and re-create it to get it going. We've checked the activemq_msgs table and could only find three entries in the DLQ. Our DEVs can't seem to find anything suspicious about the messages.
activemq version 5.15.8 Postgres version 10.7 Error WARN | Could not create JDBC tables; they could already exist. Failure was: INSERT INTO ACTIVEMQ_LOCK(ID) VALUES (1) Message: ERROR: duplicate key value violates unique constraint "activemq_lock_pkey" Detail: Key (id)=(1) already exists. SQLState: 23505 Vendor code: 0 WARN | Failure details: ERROR: duplicate key value violates unique constraint "activemq_lock_pkey" Detail: Key (id)=(1) already exists. org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "activemq_lock_pkey" Detail: Key (id)=(1) already exists. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2094)[postgresql-jdbc.jar:] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1827)[postgresql-jdbc.jar:] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)[postgresql-jdbc.jar:] at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:508)[postgresql-jdbc.jar:] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:370)[postgresql-jdbc.jar:] at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:362)[postgresql-jdbc.jar:] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:] at java.base/java.lang.reflect.Method.invoke(Method.java:566)[:] at org.postgresql.ds.jdbc23.AbstractJdbc23PooledConnection$StatementHandler.invoke(AbstractJdbc23PooledConnection.java:453)[postgresql-jdbc.jar:] at com.sun.proxy.$Proxy8.execute(Unknown Source)[:] at org.apache.activemq.store.jdbc.adapter.DefaultJDBCAdapter.executeStatement(DefaultJDBCAdapter.java:114)[activemq-jdbc-store-5.15.8.jar:5.15.8] at org.apache.activemq.store.jdbc.adapter.DefaultJDBCAdapter.doCreateTables(DefaultJDBCAdapter.java:92)[activemq-jdbc-store-5.15.8.jar:5.15.8] at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.init(JDBCPersistenceAdapter.java:307)[activemq-jdbc-store-5.15.8.jar:5.15.8] at org.apache.activemq.broker.LockableServiceSupport.preStart(LockableServiceSupport.java:89)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:54)[activemq-client-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter(BrokerService.java:687)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.startPersistenceAdapter(BrokerService.java:671)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.start(BrokerService.java:635)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)[activemq-spring-5.15.8.jar:5.15.8] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:] at java.base/java.lang.reflect.Method.invoke(Method.java:566)[:] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1763)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1700)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1630)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)[xbean-spring-4.2.jar:4.2] at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)[xbean-spring-4.2.jar:4.2] at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)[activemq-spring-5.15.8.jar:5.15.8] at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)[activemq-spring-5.15.8.jar:5.15.8] at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)[activemq-spring-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)[activemq-console-5.15.8.jar:5.15.8] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:] at java.base/java.lang.reflect.Method.invoke(Method.java:566)[:] at org.apache.activemq.console.Main.runTaskClass(Main.java:262)[activemq.jar:5.15.8] at org.apache.activemq.console.Main.main(Main.java:115)[activemq.jar:5.15.8] INFO | Database lock driver override not found for : [postgresql_native_driver]. Will use default implementation. INFO | Attempting to acquire the exclusive lock to become the Master broker INFO | Becoming the master on dataSource: org.postgresql.ds.PGPoolingDataSource@4e2c95ee INFO | PListStore:[/opt/activemq/data/localhost/tmp_storage] started INFO | JobSchedulerStore: /opt/activemq/data/localhost/scheduler started. ERROR | Failed to start Apache ActiveMQ (localhost, null) java.io.IOException: Unknown data type: 51 at org.apache.activemq.openwire.OpenWireFormat.doUnmarshal(OpenWireFormat.java:359)[activemq-client-5.15.8.jar:5.15.8] at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:201)[activemq-client-5.15.8.jar:5.15.8] at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.getLastMessageBrokerSequenceId(JDBCPersistenceAdapter.java:266)[activemq-jdbc-store-5.15.8.jar:5.15.8] at org.apache.activemq.broker.region.DestinationFactoryImpl.getLastMessageBrokerSequenceId(DestinationFactoryImpl.java:147)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.region.RegionBroker.<init>(RegionBroker.java:130)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.jmx.ManagedRegionBroker.<init>(ManagedRegionBroker.java:108)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.createRegionBroker(BrokerService.java:2399)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.createRegionBroker(BrokerService.java:2391)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.createBroker(BrokerService.java:2348)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.getBroker(BrokerService.java:1045)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.getAdminConnectionContext(BrokerService.java:2619)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.startVirtualConsumerDestinations(BrokerService.java:2780)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.startDestinations(BrokerService.java:2610)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.doStartBroker(BrokerService.java:739)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.startBroker(BrokerService.java:733)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerService.start(BrokerService.java:636)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:73)[activemq-spring-5.15.8.jar:5.15.8] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:] at java.base/java.lang.reflect.Method.invoke(Method.java:566)[:] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1763)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1700)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1630)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE] at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)[xbean-spring-4.2.jar:4.2] at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)[xbean-spring-4.2.jar:4.2] at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)[activemq-spring-5.15.8.jar:5.15.8] at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)[activemq-spring-5.15.8.jar:5.15.8] at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)[activemq-spring-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)[activemq-broker-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.15.8.jar:5.15.8] at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)[activemq-console-5.15.8.jar:5.15.8] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:] at java.base/java.lang.reflect.Method.invoke(Method.java:566)[:] at org.apache.activemq.console.Main.runTaskClass(Main.java:262)[activemq.jar:5.15.8] at org.apache.activemq.console.Main.main(Main.java:115)[activemq.jar:5.15.8] INFO | Apache ActiveMQ 5.15.8 (localhost, null) is shutting down INFO | Connector ssl stopped INFO | JobSchedulerStore: /opt/activemq/data/localhost/scheduler stopped. INFO | PListStore:[/opt/activemq/data/localhost/tmp_storage] stopped INFO | Apache ActiveMQ 5.15.8 (localhost, null) uptime 1.983 seconds INFO | Apache ActiveMQ 5.15.8 (localhost, null) is shutdown INFO | Closing org.apache.activemq.xbean.XBeanBrokerFactory$1@2ea41516: startup date [Wed Oct 23 06:57:03 UTC 2019]; root of context hierarchy WARN | Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in class path resource [activemq.xml]: Invocation of init method failed; nested exception is java.io.IOException: Unknown data type: 51 ERROR: java.lang.RuntimeException: Failed to execute start task. Reason: java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext java.lang.RuntimeException: Failed to execute start task. Reason: java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:91) at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63) at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154) at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63) at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.activemq.console.Main.runTaskClass(Main.java:262) at org.apache.activemq.console.Main.main(Main.java:115) Caused by: java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext -- Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
