Hi
Thanks. I suspect its because this code creates a new input stream in
the message body
https://github.com/apache/camel/blob/master/components/camel-zipfile/src/main/java/org/apache/camel/dataformat/zipfile/ZipIterator.java#L106
That is not explicit closed.
Can you try changing
answer.setBody(new ZipInputStreamWrapper(zipInputStream));
To
answer.setBody(zipInputStream);
And give it a test on your Windows system?
On Tue, Sep 2, 2014 at 3:13 PM, Sven Nold <[email protected]> wrote:
> Hi,
>
> same result, but workaround is to use zipSplitter like this:
>
> <bean id="zipSplitter"
> class="org.apache.camel.dataformat.zipfile.ZipSplitter" />
>
> <route id="readZip" streamCache="true">
> <from
> uri="file://target?fileName=any.xxx&moveFailed=.error&move=.done&delay=5000"/>
> <!-- <unmarshal ref="zipFileDataFormat"/> -->
> <camel:split streaming="true">
> <!-- <camel:simple>${body}</camel:simple> -->
> <camel:ref>zipSplitter</camel:ref>
> <camel:to
> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
> <camel:to uri="file://target/unzipped"/>
> </camel:split>
> <camel:to
> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
> </route>
>
> ZipSplitter seems not to have this issue. Also works for 2.13.2. which could
> be helpful for you @Thomas.
>
> So from my point of view CAMEL-7577 is not RESOLVED.
>
> Best regards,
>
> -----Ursprüngliche Nachricht-----
> Von: Claus Ibsen [mailto:[email protected]]
> Gesendet: Dienstag, 2. September 2014 14:29
> An: [email protected]
> Betreff: Re: camel 2.13.2 zipdataformat
>
> You need to build the code to make sure the snapshot is using recent code.
>
> On Tue, Sep 2, 2014 at 2:22 PM, Sven Nold <[email protected]> wrote:
>> Hi Claus,
>>
>> I changed to 2.13.3-SNAPSHOT but still the same issue.
>>
>> Best regards,
>>
>> Sven
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Claus Ibsen [mailto:[email protected]]
>> Gesendet: Dienstag, 2. September 2014 13:52
>> An: [email protected]
>> Betreff: Re: camel 2.13.2 zipdataformat
>>
>> Hi
>>
>> See CAMEL-7577
>>
>> On Tue, Sep 2, 2014 at 1:49 PM, Sven Nold <[email protected]> wrote:
>>> Hi,
>>>
>>> interesting is also that if I only have on zipEntry and define the
>>> zipDataFormat as useIterator=false it works fine. So my guess is that
>>> ZipIterator somehow holds a lock on the file:
>>>
>>> <bean id="zipFileDataFormat"
>>> class="org.apache.camel.dataformat.zipfile.ZipFileDataFormat">
>>> <property name="usingIterator" value="false"/>
>>> </bean>
>>>
>>> <bean id="zipAggregationStrategy"
>>> class="org.apache.camel.processor.aggregate.zipfile.ZipAggregationStr
>>> a
>>> tegy"/>
>>>
>>> <camelContext xmlns="http://camel.apache.org/schema/spring";>
>>> <!-- write the zip so we can process something repeatedly -->
>>> <route id="writeZip">
>>> <from
>>> uri="file://src/main/data?antInclude=message1.xml&noop=true"/>
>>> <aggregate strategyRef="zipAggregationStrategy"
>>> completionFromBatchConsumer="true" eagerCheckCompletion="true"
>>> completionSize="2">
>>> <correlationExpression>
>>> <constant>true</constant>
>>> </correlationExpression>
>>> <to uri="file://target?fileName=any.xxx"/>
>>> </aggregate>
>>> </route>
>>>
>>> <!-- extract the zip -->
>>> <route id="readZip" streamCache="true">
>>> <from
>>> uri="file://target?fileName=any.xxx&moveFailed=.error&move=.done&delay=5000"/>
>>> <unmarshal ref="zipFileDataFormat"/>
>>> <!--<camel:split streaming="true">
>>> <camel:simple>${body}</camel:simple>
>>> <camel:to
>>> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
>>> <camel:to uri="file://target/unzipped"/>
>>> </camel:split> -->
>>> <camel:to
>>> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
>>> </route>
>>> </camelContext>
>>>
>>> Works fine.
>>>
>>> Best regards,
>>>
>>>
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Sven Nold [mailto:[email protected]]
>>> Gesendet: Dienstag, 2. September 2014 13:38
>>> An: [email protected]
>>> Betreff: AW: camel 2.13.2 zipdataformat
>>>
>>> Hi,
>>>
>>> thanks for this test now I think here is the problem:
>>>
>>> [l-1) thread #1 - file://target] GenericFileOnCompletion WARN Error
>>> during commit. Exchange[any.zip]. Caused by:
>>> [org.apache.camel.component.file.GenericFileOperationFailedException -
>>> Error renaming file from
>>> C:\temp\camel-examples-master\zip-file-processing\target\any.zip to
>>> target\.done\any.zip]
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Error
>>> renaming file from
>>> C:\temp\camel-examples-master\zip-file-processing\target\any.zip to
>>> target\.done\any.zip
>>> at
>>> org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:77)
>>> at
>>> org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:113)
>>> at
>>> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:88)
>>> at
>>> org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:124)
>>> at
>>> org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:80)
>>> at
>>> org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:54)
>>> at
>>> org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:100)
>>> at
>>> org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:228)
>>> at
>>> org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:61)
>>> at
>>> org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:613)
>>> at
>>> org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:581)
>>> at
>>> org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:240)
>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:106)
>>> at
>>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>>> at
>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:423)
>>> at
>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211)
>>> at
>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175)
>>> at
>>> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187)
>>> at
>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.io.IOException: Renaming file from
>>> 'C:\temp\camel-examples-master\zip-file-processing\target\any.zip' to
>>> 'target\.done\any.zip' failed: Cannot delete file
>>> 'C:\temp\camel-examples-master\zip-file-processing\target\any.zip' after
>>> copy succeeded
>>> at
>>> org.apache.camel.util.FileUtil.renameFileUsingCopy(FileUtil.java:456)
>>> at org.apache.camel.util.FileUtil.renameFile(FileUtil.java:428)
>>> at
>>> org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:74)
>>> ... 25 more
>>>
>>> This is why it grabs the file all the time.
>>>
>>> Regarding your questions:
>>>
>>> 1) well thanks to your test you can see error above
>>> 2) yes it is processing it over and over again
>>> 3) no doesn't move to .error
>>> 4) no file is not updated, in your test case I am not quite sure, will
>>> increase delay or as you suggested preMove option.
>>>
>>> Due to known problems off windows and zip files I even renamed it to .xxx
>>> to keep away explorer from analyzing, but still it seems there is a handle
>>> on that file.
>>> And again if I remove the unmarshal and split it works fine.
>>>
>>> Thanks for any help.
>>>
>>> Best regards,
>>>
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Jimmi Dyson [mailto:[email protected]]
>>> Gesendet: Dienstag, 2. September 2014 10:37
>>> An: [email protected]
>>> Betreff: Re: camel 2.13.2 zipdataformat
>>>
>>> Hi,
>>>
>>> I just added an example at
>>> https://github.com/jimmidyson/camel-examples/tree/master/zip-file-processing.
>>> This example creates the zip file first (so that it's repeatable) & then
>>> does pretty much what you're doing: read from file uri, unmarshal, log each
>>> file contents (careful with large files) & finally writes each file to a
>>> file. Seems to be doing what it should...
>>>
>>> Few questions if the example doesn't help you:
>>>
>>> - Are there any hints in the logs (try increasing logging level if not)?
>>> - If the file is back in the source dir does it try to process again?
>>> - Is there anything in the .error (moveFailed dir)?
>>> - Could the source file be being updated while you're processing it - if
>>> so, can you try the preMove option?
>>>
>>> Thanks,
>>>
>>>
>>> On 2 September 2014 08:46, Walzer, Thomas
>>> <[email protected]>
>>> wrote:
>>>
>>>> Hi Sven,
>>>>
>>>> I seem to have a similar problem. I also use 2.13.2.
>>>> My code is almost identical. The only difference is my in-url, that
>>>> looks
>>>> like:
>>>>
>>>> file:////someserver/somedir/in?move=archive/${date:now:yyyyMMdd}/${f
>>>> i l e:name}.${date:now:HHmmss}&antInclude=*.ZIP
>>>>
>>>> I get a random number of retrievals (about 4-20) and therefore the
>>>> same number of files in the „archive“ directory. The timestamps are
>>>> 6 seconds apart.
>>>>
>>>> Cheers, Thomas.
>>>>
>>>> Am 02.09.2014 um 08:44 schrieb Sven Nold <[email protected]>:
>>>>
>>>> > Hi Guys,
>>>> >
>>>> > I am trying to read Zipentries and process them separately. But
>>>> > using
>>>> the file component it seems the file is not correctly closed. I can
>>>> see the file for a few seconds in the .done and source directory.
>>>> But then it disappears in the .done folder and is back in the source
>>>> directory :
>>>> >
>>>> >
>>>> > <bean id="zipFileDataFormat"
>>>> class="org.apache.camel.dataformat.zipfile.ZipFileDataFormat">
>>>> > <property name="usingIterator" value="true" /> </bean>
>>>> >
>>>> > <route id="readZip">
>>>> > <from
>>>> uri="file://C:/temp/?fileName=any.zip&moveFailed=.error&move=.done&delay=10000"
>>>> />
>>>> > <unmarshal ref="zipFileDataFormat" />
>>>> > <!-- <split streaming="true"> -->
>>>> > <!-- <simple>$simple{body}</simple> -->
>>>> > <!-- <to
>>>> uri="log://OUT?level=INFO&showAll=true&multiline=true" />
>>>> -->
>>>> > <!-- </split> -->
>>>> > <to uri="log://OUT?level=INFO&showAll=true&multiline=true"
>>>> />
>>>> > </route>
>>>> >
>>>> > If I remove the unmarshal the file component works fine (it moves
>>>> processed file to .done).
>>>> >
>>>> > Any Ideas? I guess I could do a workaround by using noop=true and
>>>> > moving
>>>> the file myself, but it would be nice if it works as expected.
>>>> >
>>>> > Best regards,
>>>> >
>>>> > Sven
>>>>
>>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> Red Hat, Inc.
>> Email: [email protected]
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
>> hawtio: http://hawt.io/
>> fabric8: http://fabric8.io/
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: [email protected]
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
> hawtio: http://hawt.io/
> fabric8: http://fabric8.io/
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: [email protected]
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
