Hello, I'm using Apache Fuse ESB with Apache Camel 2.4.0 to process some large files. The ActiveMQ 5.5.0 broker comes with ServiceMix and it's almost a basic configuration. The architecture looks something like this: 1. service unit that reads lines from a file and sends each line within the exchange's out body 2. service unit 2 that takes the line from the previous exchange, transforms it into an xml and sends the xml within the exchange's out body 3. service unit 3 that takes the xml, transforms it into another xml and sends the xml 4. service unit 4 that takes the final xml, unmarshals it into an object, inserts the object into the db and sends a final exchange to a service that checks if it's the end of the file or not 5. service unit 5 that resets a flag if it's the end of the file
This architecture was working fine but had a "minor" problem, the insert/update operations into the db(service unit 4) were extremely slow, it would take more than 3 hours to handle 18.000 lines from the initial file. In the end I found out that there is a parameter that must be set "sendStringParametersAsUnicode=false" in order to improve the performance when working with Sql Server 2000+. After setting this parameter on the jdbc driver url the performance did increase dramatically, the db operations would take less than 1 second, from more than 50 seconds before. But an exception showed up and I have no idea how to solve it. This is the current configuration for the queues that Apache Camel uses to communicate from one service unit to the next one: <amq:policyEntry queue=">" producerFlowControl="true" memoryLimit="256kb"> With this configuration, when I add "sendStringParametersAsUnicode=false" I get the EOFException after integrating about 7000 lines. If I switch it to: <amq:policyEntry queue=">" producerFlowControl="true" memoryLimit="512mb"> I can process around 10000 lines but I still get the Exception. If I increase memoryLimit to 1mb or more I get another exception, saying that servicemix can't reach the broker on port 62626 because there is another binding on that port already, which is nonsense, because there is nothing for sure. I'm running the basic configuration, with 256kb, on my local machine, which is like 3 times faster, and I can't crash the broker and get any exception, no matter how much I try. He had the same problem and solved it, but I don't understand his point: http://camel.465427.n5.nabble.com/handling-large-files-tp471986p472003.html I suspect that this is an ActiveMQ problem, the broker simply closes a connection, but I don't understand why or where. Here is the complete stacktrace: ERROR | Camel Thread 0 - file:///d:/activesmx/inbox | DefaultErrorHandler | .apache.camel.processor.Logger 248 | Failed delivery for exchangeId: 3889e287-a8aa-475c-b377-26a6963c19d2. Exhausted after delivery attempt: 1 caught: org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.io.EOFException org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.io.EOFException at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316) at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168) at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469) at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.send(JmsConfiguration.java:195) at org.apache.camel.component.jms.JmsProducer.doSend(JmsProducer.java:375) at org.apache.camel.component.jms.JmsProducer.processInOnly(JmsProducer.java:320) at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:150) at org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:50) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:104) at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:272) at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:98) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:99) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:290) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202) at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.Pipeline.process(Pipeline.java:143) at org.apache.camel.processor.Pipeline.process(Pipeline.java:78) at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:113) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:61) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:104) at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:272) at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:98) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:99) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:290) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202) at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.Pipeline.process(Pipeline.java:143) at org.apache.camel.processor.Pipeline.process(Pipeline.java:78) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:290) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:334) at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:274) at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:179) at org.apache.camel.processor.Splitter.process(Splitter.java:93) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:99) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:290) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202) at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.Pipeline.process(Pipeline.java:143) at org.apache.camel.processor.Pipeline.process(Pipeline.java:78) at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:113) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:61) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:104) at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:272) at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:98) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:99) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:290) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202) at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:72) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:99) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:290) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:202) at org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:256) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.Pipeline.process(Pipeline.java:143) at org.apache.camel.processor.Pipeline.process(Pipeline.java:78) at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322) at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:153) at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:119) at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:98) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source) at java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: javax.jms.JMSException: java.io.EOFException at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49) at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1296) at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1393) at org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:310) at org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196) at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:457) ... 144 more Caused by: java.io.EOFException at java.io.DataInputStream.readInt(Unknown Source) at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:276) at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:230) at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:222) at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:205) ... 1 more -- View this message in context: http://activemq.2283324.n4.nabble.com/javax-jms-JMSException-java-io-EOFException-in-ActiveMQ-tp4664311.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
