[
https://issues.apache.org/jira/browse/CAMEL-11656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16262500#comment-16262500
]
ASF GitHub Bot commented on CAMEL-11656:
----------------------------------------
GitHub user onders86 opened a pull request:
https://github.com/apache/camel/pull/2108
CAMEL-11656 - add preSort option
even though i believe this issue is not related to sorting file list
beforehand, i am just opening the PR to discuss further and revise...
@davsclaus could you review?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/onders86/camel CAMEL-11656
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/camel/pull/2108.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2108
----
commit b7e51f1a3c1aec4be3b5ae546a8ef74ef25f3e7b
Author: onders86 <[email protected]>
Date: 2017-11-22T11:52:41Z
CAMEL-11656 - add preSort option
----
> Support default directory sorter for FileConsumer
> -------------------------------------------------
>
> Key: CAMEL-11656
> URL: https://issues.apache.org/jira/browse/CAMEL-11656
> Project: Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.19.0
> Reporter: Viktor Khoroshko
> Priority: Minor
> Fix For: 2.21.0
>
>
> Hello,
> The current way of sorting consumed files isn't very flexible due to how it's
> implemented:
> Files are sorted AFTER they're added to in progress repository what in
> combination with *maxMessagesPerPoll* set to be > 0 and
> *eagerLimitMaxMessagesPerPoll* set to false forces the
> *removeExcessiveInProgressFiles* call - which in case of persistent
> idempotent repository used causes redundant calls to a database.
> This is not only the issue.
> In my case I have a logic in a custom file filter that a file shouldn't be
> processed if a file with a same prefix is already in progress while still
> those files should be processed in a specified order.
> The issue is that since sorting is performed after each file is added to in
> progress repository and before added there a file should be accepted by a
> filter but it will be not - as a previous file with a same prefix has
> already been added to in progress repo.
> *Example*: files - test.001, test.002, test.003
> If test.003 is first in a returned file list then it will be added to in
> progress repository and next files will not be accepted due to the same
> prefix.
> Default sorting would solve this issue.
> It would be great if default sorter can be specified for *FileConsumer* in
> the pollDirectory method:
> {code:java}
> log.trace("Polling directory: {}", directory.getPath());
> File[] dirFiles = directory.listFiles();
> if (dirFiles == null || dirFiles.length == 0) {
> // no files in this directory to poll
> if (log.isTraceEnabled()) {
> log.trace("No files found in directory: {}",
> directory.getPath());
> }
> return true;
> } else {
> // we found some files
> if (log.isTraceEnabled()) {
> log.trace("Found {} in directory: {}", dirFiles.length,
> directory.getPath());
> }
> }
> List<File> files = Arrays.asList(dirFiles);
> // sort there?
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
