We have the following requirement: 1. Poll from a database 2. Marshall the result set into XML 3. Send the XML in a JMS message to a downstream system
The issue is, some of the result sets could be huge - millions of rows. The transformer may be unable to handle this simply because of memory/heap space limitations. There is also the issue of sending a massive message. Does Camel have an easy solution for this, some way of streaming the result set to a transformer in small chunks, then sending individual messages for each piece which are identified as part of a collection by some kind of correlation id or something similar? Or is our requirement simply not suited for messaging infrastructure, in which case we should look at a solution based around batch processing? Thanks -- View this message in context: http://camel.465427.n5.nabble.com/Streaming-huge-files-in-Camel-tp5750096.html Sent from the Camel - Users mailing list archive at Nabble.com.
