[
https://issues.apache.org/jira/browse/CAMEL-24575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115673#comment-18115673
]
Mani commented on CAMEL-24575:
------------------------------
Hi team, following up on the same — wanted to check if this is something that's
feasible or fits within the roadmap. Happy to provide any additional context if
it would help move the evaluation forward.
> SFTP Consumer: Request for lock-strategy-agnostic minimum-age file filtering
> ----------------------------------------------------------------------------
>
> Key: CAMEL-24575
> URL: https://issues.apache.org/jira/browse/CAMEL-24575
> Project: Camel
> Issue Type: Improvement
> Components: camel-ftp, camel-sftp
> Affects Versions: 4.14.5
> Reporter: Mani
> Priority: Major
>
> *Background:*
> CAMEL-23931 introduced preSort=modified which resolved our oldest-first
> ordering requirement beautifully. Thank you for the quick turnaround on that.
> This ticket is a follow-up for a related gap we are still working around.
>
> *Our Use Case:*
> We need the SFTP consumer to handle two guarantees when picking up files:
> * Process files in oldest-modified-first order — resolved by preSort=modified
> * Skip files that are still actively being written to the server — only pick
> up files that have been stable for a minimum duration
>
> *The Gap:*
> Camel already has readLockMinAge which is exactly the semantic we need for
> requirement 2. However, in our setup we use a custom lock strategy with
> readLock=none and readLockMinAge appears to be unavailable unless
> readLock=changed is configured. This means we cannot benefit from it
> regardless of what we set.
>
> *Current Workaround:*
> We maintain a size-snapshot tracker inside our custom lock strategy
> implementation that records file sizes across consecutive poll cycles and
> only allows processing once the size has been unchanged for the configured
> duration.
>
> *The Ask:*
> Would it be feasible to expose a min-age filter at the same level as preSort
> — operating on the SftpRemoteFile[] array before any lock strategy is
> involved? Since this sits upstream of lock acquisition it would be
> lock-strategy-agnostic and works alongside any custom locking. Combined with
> preSort=modified this would give us oldest-stable-file-first natively and let
> us remove all custom stability-tracking.
>
>
> *References:*
> CAMEL-23931 — preSort feature (related prior request)
> Mailing list thread ([email protected], August 12):
> [link|https://lists.apache.org/thread/3nwv3rh5ffjl5v5x08hprq3w0qckn28o]
>
> Tagging [~davsclaus] for visibility
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
