shashank created CAMEL-25549:
--------------------------------
Summary: camel-atom - with sortEntries=true the consumer routes
the entries newest first, and with the default idempotent strategy only the
newest entry of a feed is routed
Key: CAMEL-25549
URL: https://issues.apache.org/jira/browse/CAMEL-25549
Project: Camel
Issue Type: Bug
Components: camel-atom
Reporter: shashank
Assignee: shashank
{{AtomUtils.readItems}} (lines 44-60 at main 9432ba16d026) sorts the items of a
feed with {{ItemComparator.oldestItemFirst()}} when {{sortEntries=true}}.
{{FeedEntryPollingConsumer.poll}} routes the entries from the end of the list
({{entryIndex = list.size() - 1}}, {{list.get(entryIndex--)}}), so the sorted
entries are routed from the newest to the oldest one. camel-rss sorts newest
first for the same loop ({{RssDateComparator}}) and routes oldest first
({{RssEntrySortTest}} expects ascending dates), and so did camel-atom up to 3.x
({{AtomEntrySortTest}}, removed when the parser was replaced in 4.0).
Since 4.21 the consumer is idempotent by default (CAMEL-23220):
{{ItemUpdatedIdempotentStrategy}} routes an entry only when its updated (or
published) date is after the date of the entry routed before it. With the
newest entry routed first, every other entry of the feed is older and is
skipped. So
{code:java}
from("atom:https://example.org/feed?sortEntries=true";).to("...");
{code}
routes one entry of the feed, the newest, and silently skips all the others; at
later polls, of several new entries only the newest is routed. With
{{idempotent=false}} all entries are routed, newest first.
h3. Reproduction
New {{AtomEntrySortTest}}: one poll of {{src/test/data/unsortedfeed.atom}} (10
entries), {{splitEntries=true&throttleEntries=false}}, the consumer polled
directly. On main, twice:
{noformat}
sortedEntriesAreAllRoutedOldestFirst: All the entries of the feed must be
routed ==> expected: <10> but was: <1>
sortedEntriesAreRoutedOldestFirstWithoutIdempotency: expected:
<[2008-08-27T08:09:43-05:00, ..., 2008-11-12T08:08:35-06:00]> but was:
<[2008-11-12T08:08:35-06:00, ..., 2008-08-27T08:09:43-05:00]>
{noformat}
A third test (the same feed without {{sortEntries}}: 10 entries, oldest first)
passes on main too (the change does not touch that path).
h3. Proposed fix
{{AtomEntryPollingConsumer}} reverses the sorted list, so that the loop, which
starts at the end, routes the oldest entry first. {{AtomUtils.readItems}} and
the list that the consumer with {{splitEntries=false}} puts in the body are
unchanged. 4.23 upgrade guide note added and marked as a breaking change
(routes with {{sortEntries=true&idempotent=false}} now get the entries in the
reverse order, oldest first). camel-atom with the fix: 36 tests, 0 failures, 1
skipped (as on main).
For information, not changed: since 4.0 the entries are sorted by date also
without {{sortEntries}} ({{Comparator.naturalOrder()}} of the parser's items is
newest first, routed from the end: oldest first), so after the fix both
settings route in the same order.
Found with a Lean 4 model of the poll (an entry as its date, the idempotent
strategy as "keep an entry after the last kept date"): for every sorted feed
main is proved to route only the newest entry, so it loses entries whenever the
feed has two or more; the fix is proved to route every entry of a sorted feed,
in order. Confirmed with the real consumer.
Affected: the reversed order in main, camel-4.22.x, camel-4.18.x, camel-4.14.x
(same {{AtomUtils}}, GitHub contents API; since 4.0.0); the skipped entries in
main and camel-4.22.x (the idempotent strategy exists since 4.21.0).
Duplicate check (2026-10-11): JIRA text "sortEntries" (4: CAMEL-1156 added the
option in 2008, CAMEL-7784 is about camel-rss entries with identical dates),
summary "camel-atom" since 2023-06 (6, none about the order); GitHub pull
requests with "atom" in the title: none open. No open pull request changes
{{AtomEntryPollingConsumer}} or {{AtomUtils}}.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
