Hi,
In my first attempt to use Camel I’ve run into a intra-route timing issue that
I’ve only solved with a hack, so I was wondering whether there are any best
practices of dealing with timing issues when dealing with multiple processing
steps in a batch file pipeline.
Basically I am trying to avoid doing an HTTP POST with an empty payload, since
the route performing the HTTP POST is triggered before the file that it is
wired to upload has been written.
If I turn stream caching on, this problem goes away. However, since some files
can be quite big, I’d prefer not to have do stream caching.
So to solve the issue, I’ve written a workaround bean that just does a
Thread.sleep() in order to wait for the upload file to actually get some data
in it before firing off the HTTP POST.
I’ve got a two step pipeline that:
1. Transcodes a batch input file into an intermediate format (using msgpack
serialization);
2. Performs an HTTP POST of the intermediate format to a remote server;
I’d like to keep the intermediate format around on disk for debugging and
manual replay tasks.
My camel context has two routes:
<route id=“transcode-to-msgpack">
<from uri="file:/tmp/d"/>
<log message="Transcoding ${file:name} to msgpack" />
<to uri="bean:transcoder"/>
<to uri="file:/tmp/b?fileName=${file:name.noext}.msgpack"/>
</route>
<route id=“post-msgpack-payload">
<from uri="file:/tmp/b"/>
<from uri="file:/tmp/e"/>
<log message="POSTing ${file:name} to the rating API" />
<setProperty propertyName="url.template">
<constant>http://localhost:9999/calls/:source/:sequence</constant>
</setProperty>
<process ref=“httpDataPump"/>
</route>
I have two custom beans doing the work:
1. transcoder - This takes an InputStream, and returns an InputStream that
wraps and transcodes the InputStream from the file;
2. httpDataPump - This contains an HTTP client that uploads the FileInputStream
the the InMessage from the Exchange refers.
Doing a Thread.sleep() seems like a real hack to me, so I was wondering if
there is a more idiomatic way to solve the issue. I’ve looked into the
preMoveNamePrefix options, but they appear to apply only to input files.
Ideally I’m looking for something that can move the output file after it has
been written.
Any pointers are appreciated.
Cheers,
Ben
