Re: Write enriched data after a split in a file

[Jean Francois LE BESCONT]

Thanks Willem for help.

My explaination are sometime to complicate, I will present it differently.

I have a "big" file to process efficiently, for that if I have tried :

Method 1 :
   from("file://C:/Temp/camel/input_test/?noop=true")
      .split()
         .tokenize("\n")
            // Business logic with possible reject and enrich
            .recipientList(
simple("stream:file?fileName=C:/Temp/camel/output_test/out.csv"))
            .end()
      .end()
   .end();

With this method, my process take 11 seconds but my file is not released
(windows file is locked in windows).

So I try with append :

Method 2 :

   from("file://C:/Temp/camel/input_test/?noop=true")
      .split()
         .tokenize("\n")
            // Business logic with possible reject and enrich

.to("file://?fileName=C:/Temp/camel/output_test/out.csv&fileExist=Append")
      .end()

   .end();

File is not locked after but it take 1,40 minutes ...

I finally try with the aggregate :

Method 3 :

   from("file://C:/Temp/camel/input_test/?noop=true")
      .split()
         .tokenize("\n")
            // Business logic with possible reject and enrich
            .aggregate(header(Exchange.FILE_NAME_ONLY), new
BodyInAggregatingStrategy())
               .completionTimeout(3000)
               .to("file://C:/Temp/camel/output_test/")
            .end()
      .end()
   .end();

As the example, but the process never finish, so I try to specify
completionSize(7500) and  completionTimeout(15000) but my output file
doesn't contains all my row ( only 1 %). I have not find an example of how
to use these parameters ...

Please, what should I do to have a process efficient and with no lock ?

Thanks

JF