shashank created CAMEL-25684:
--------------------------------
Summary: camel-slack - the consumer skips the messages of a poll
it did not route when the route is suspended during that poll
Key: CAMEL-25684
URL: https://issues.apache.org/jira/browse/CAMEL-25684
Project: Camel
Issue Type: Bug
Components: camel-slack
Reporter: shashank
Assignee: shashank
{{SlackConsumer.createExchanges}} moves {{timestamp}} to the most recent
message of the poll (lines 152 / 161 at main cd2ce899764e) before
{{processBatch}} routes the messages. {{processBatch}} (172-192) stops when
{{isBatchAllowed()}} is false, which {{ScheduledBatchPollingConsumer}} (104)
returns when the route is being suspended ({{RouteController.suspendRoute}},
also from a route policy that suspends the route through the route controller
such as {{DurationRoutePolicy}}, or {{CamelContext.suspend()}}, which resumes
the same consumer): the message that is being routed completes and the rest of
the batch is not routed. When the route is resumed the same consumer asks Slack
for the messages after {{timestamp}} only, so the messages of that poll that
were not routed are never routed. Nothing is logged. With the defaults the most
recent message is routed first, so the older messages of the poll are lost: the
rest of that poll, which since CAMEL-25523 can be several pages of
{{maxResults}} (default 10) messages each.
This is the shape that CAMEL-25570 fixed in aws2-ddbstream. A consumer
suspended directly ({{consumer.suspend()}}, as {{ScheduledRoutePolicy}} and the
throttling route policies do) does not end the batch and is not affected; a
route that is stopped and started (a new consumer) is not affected either.
h3. Reproduction
New {{SlackConsumerSuspendedBatchTest}} (4 tests): a real route against a small
local HTTP server that answers {{conversations.history}} as Slack does
(messages after {{oldest}}, most recent first). The processor of one message
suspends the route from another thread and waits until the batch has ended;
then the route is resumed; the fourth test suspends and resumes the
{{CamelContext}}. On main all 4 fail: {{expected: <[m5, m4, m3, m2]> but was:
<[m5, m4]>}}, in natural order {{expected: <[m2, m3, m4, m5]> but was: <[m2,
m3]>}}, with a message posted while suspended {{expected: <[m5, m6, m4, m3,
m2]> but was: <[m5, m6]>}}, and for the context suspend {{expected: <[m5, m4,
m3, m2]> but was: <[m5, m4]>}}. The same tests with the pooled exchange
factory, plus a check of its statistics (the exchanges of the messages that
were not routed are given back once), fail on main as well.
h3. Proposed fix
* The messages that were not routed are kept and routed by the next poll,
together with the messages posted in the meantime, in the order of the
endpoint. The timestamp cannot simply stay before them as in the ddbstream fix:
by default the most recent message is routed first, so the messages that were
not routed are older than the ones that were, and a timestamp before them would
route those again.
* As in the ddbstream fix, the exchanges of those messages are given back to
the exchange factory, and {{processBatch}} returns the number of exchanges it
routed (used for {{sendEmptyMessageWhenIdle}} and greedy polling) instead of
the number it was given.
* The timestamp is now set in {{poll()}} instead of in {{createExchanges}}
(same value).
* An upgrade guide entry (4.24). The kept messages are in memory only: a route
that is stopped and started again does not route them (a new consumer).
camel-slack tests with the fix (built with the current core modules): 19 run, 0
failures, 1 skipped (as on main).
How it was found: a TLA+ model of the poll (fetch, route one message at a time
while the batch is allowed, end of the batch, suspend and resume as environment
steps) violates "between two polls the consumer has not moved past a message it
did not route" on main (5 states), holds with the change (newest-first and
natural order); then the test.
Affected: main (cd2ce899764e; tested), camel-4.23.x, camel-4.22.x,
camel-4.18.x, camel-4.14.x (the same {{createExchanges}} / {{processBatch}};
read, not run).
Duplicate check (2026-10-15): JIRA component camel-slack, text "slack consumer
suspend" (CAMEL-25523, reading the following pages of the history, already
fixed - a different cause of skipped messages); GitHub pull requests
"SlackConsumer" (#27674 = CAMEL-25523, merged; nothing open).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
