shashank created CAMEL-25679:
--------------------------------
Summary: camel-mongodb - the tailable and change streams consumers
stop consuming for good (route still started) when the server is unreachable
for longer than the server selection timeout
Key: CAMEL-25679
URL: https://issues.apache.org/jira/browse/CAMEL-25679
Project: Camel
Issue Type: Bug
Components: camel-mongodb
Reporter: shashank
Assignee: shashank
The tailable cursor consumer ({{consumerType=tailable}}, the default) and the
change streams consumer ({{consumerType=changeStreams}}) read from a cursor on
one thread, {{MongoAbstractConsumerThread.run()}}. When reading fails (for
example the connection to the server is lost), the loop logs a WARN ("Will try
again on next poll"), closes the cursor, waits {{cursorRegenerationDelay}}
(1000 ms by default) and opens a new cursor ({{initializeCursor()}}).
At main cd2ce899764e the new cursor is opened at line 130, after the {{try}} of
the loop (101-115). While the server is still unreachable, opening the cursor
fails: the driver gives up after its server selection timeout (30 seconds by
default) with {{MongoTimeoutException: Timed out while waiting for a server}}.
The exception leaves {{run()}}, the consumer thread ends, and:
* the route stays started and the consumer never reads again, also long after
the server is back; only a restart of the route helps;
* the only trace is the uncaught exception printed on stderr by the thread
({{Exception in thread "Camel (camel-1) thread #1 - mongodb://..."
com.mongodb.MongoTimeoutException ...}}), not through the logging framework.
Trigger: default options, a MongoDB server (or replica set) that cannot be
reached for longer than about 31 seconds (server selection timeout plus the
regeneration delay): a restart, a failover that takes long, a network outage,
maintenance. A long-running tailing or change stream consumer will see such an
outage.
h3. Reproduction
New {{MongoDbConsumerCursorRegenerationTest}} with Mockito mocks of the driver
objects (no server): the cursor of the start fails its read
({{MongoSocketReadException}}), the first new cursor cannot be opened
({{MongoTimeoutException}}), the next one returns a document. On main:
* tailable consumer: {{mock://result Received message count. Expected: <1> but
was: <0>}}, and the test output shows {{Exception in thread "Camel (camel-1)
thread #1 - mongodb://myDb" com.mongodb.MongoTimeoutException: Timed out while
waiting for a server}};
* change streams consumer: the same.
* A third test (the tailing thread run directly, stopped while it opens a new
cursor) pins a related small leak of the same lines: a cursor opened while the
consumer is being stopped was neither read nor closed on main ({{Wanted but not
invoked: mongoCursor.close()}}).
h3. Proposed fix
Open the new cursor at the start of the next iteration of the loop, inside its
{{try}}: a failure is logged like a failed read ("Exception from consuming from
MongoDB caused by ... Will try again on next poll.") and tried again after
{{cursorRegenerationDelay}}, until the server is back or the consumer is
stopped. A cursor opened while the consumer was being stopped is closed instead
of read (stop() could not close it). The {{cursor}} field becomes volatile, as
{{stop()}} reads it from another thread.
While the server is down there is one WARN with stack trace per attempt, every
server selection timeout plus {{cursorRegenerationDelay}} (about 31 seconds by
default), like the WARN that main already logs for every failed read. No option
or default changes, so no upgrade guide entry.
camel-mongodb unit tests with the fix: 21 run, 0 failures, 0 skipped. Not run:
the module's integration tests (they need a MongoDB container), a real outage.
How it was found: reading the consumers' loop during a sweep of reconnect
paths; the test confirmed it.
Not changed: the tailable consumer still ends quietly when the driver throws
{{IllegalStateException}} from {{hasNext()}} (meant for a cursor closed by
{{stop()}}); the change streams consumer logs a failed read at DEBUG only
(CAMEL-16025).
Affected: main (cd2ce899764e; tested). camel-4.23.x (line 130), camel-4.22.x,
camel-4.18.x and camel-4.14.x (line 94) open the new cursor outside the {{try}}
the same way (read, not run).
Duplicate check (2026-10-15): JIRA text "MongoAbstractConsumerThread"
(CAMEL-16025, CAMEL-16024, CAMEL-25024, CAMEL-25025, CAMEL-17121 retry logic
consolidation; none about the regeneration), "MongoTimeoutException"
(CAMEL-24257, camel export; CAMEL-12142, tests), component camel-mongodb with
"tailable cursor" (CAMEL-11977 stop on shutdown, 2017); GitHub pull requests
"MongoAbstractConsumerThread" (#26961, failure handling of the route, did not
touch the regeneration), "mongodb tailable cursor", "mongodb change streams
reconnect" (nothing related).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
