shashank created CAMEL-25609:
--------------------------------
Summary: camel-browse - browseLimit (advanced, off by default)
does not release old exchanges (the limit is a sub list view), and with
concurrent producers a stale view makes the endpoint fail with
ConcurrentModificationException until restart
Key: CAMEL-25609
URL: https://issues.apache.org/jira/browse/CAMEL-25609
Project: Camel
Issue Type: Bug
Components: camel-core
Reporter: shashank
Assignee: shashank
Scope: {{browseLimit}} on the browse endpoint is an advanced option that is off
by default (0 = keep everything), on a component used for testing and
debugging. Nothing in Camel sets it: the only caller of {{setBrowseLimit}} on a
browse endpoint is {{BrowseComponent}}, which copies its own option of the same
name (default 0). Only users who set the option are affected.
{{BrowseEndpoint.onExchange}} (lines 141-151 at main a7169d7c6ab6) makes room
for a new exchange like this:
{code:java}
if (browseLimit > 0) {
// make room if end of capacity
int max = exchanges.size();
if (max >= browseLimit) {
int space = Math.abs(max - (browseLimit - 1));
exchanges = exchanges.subList(space, max);
}
}
exchanges.add(exchange);
{code}
{{exchanges}} is a {{CopyOnWriteArrayList}} ({{createExchangeList}}).
{{subList}} returns a view of that list, and a sub list of a sub list is again
a view of the same backing list; {{add}} on the view appends to the backing
list. Nothing is ever removed from the backing list.
Two effects:
# *Exchanges are not released (reproduced).* The option says "Maximum number of
messages to keep in memory available for browsing". With
{{browse:foo?browseLimit=10}} a browser sees the last 10 exchanges, but the
endpoint holds every exchange that was ever sent to it (1000 after 1000
messages), with bodies and headers, until the endpoint is stopped. The limit
has no effect on memory, and because the list is copy-on-write every new
exchange copies the whole, growing backing array.
# *A stale view fails (reproduced).* A sub list of a {{CopyOnWriteArrayList}}
throws {{ConcurrentModificationException}} when the backing list was changed
through another view.
#* A list that {{getExchanges()}} returned can no longer be read after one more
exchange arrived ({{size()}}, {{get}}, iteration all fail). Readers such as
{{ManagedBrowsableEndpoint}} ({{getExchanges().size()}}, then {{get(index)}})
can hit this when a message arrives in between.
#* Two threads sending to the same endpoint: thread A creates its view, thread
B creates its own view and adds, then A stores its view in the field, which is
now outdated. A's {{add}} fails, the outdated view stays in the field, and from
then on every exchange sent to the endpoint ({{exchanges.size()}} in
{{onExchange}}) and every read of it fails with
{{ConcurrentModificationException}} until the endpoint is restarted. Reproduced
with a test that holds thread A exactly between creating and storing the view
(not a load test). This failure is loud: the producer throws and the route's
error handling sees it.
A smaller point in the same lines: the view (with {{browseLimit - 1}} entries)
is stored before the add, so a reader can see one exchange fewer than the limit
while an exchange arrives, and an empty endpoint with {{browseLimit=1}}
({{BrowsableEndpoint.getBrowseStatus}} reports size 0). Reproduced.
Without {{browseLimit}} the endpoint keeps everything by design and none of
this applies.
h3. Reproduction
{{BrowseLimitMemoryTest}} in camel-core, 5 tests; all 5 fail on main:
* {{testOldExchangesAreReleased}}: a {{BrowseEndpoint}} with {{browseLimit=10}}
whose {{createExchangeList}} hands the created list to the test; 1000 messages.
The browser sees the last ten on main and with the fix; the list that holds the
exchanges must have 10 entries. On main: {{expected: <10> but was: <1000>}}.
* {{testBrowsedListStaysUsableWhenMoreExchangesArrive}}:
{{browse:live?browseLimit=2}}, three messages, {{getExchanges()}}, one more
message, then {{size()}} of the list from before. On main:
{{java.util.ConcurrentModificationException}}.
* {{testSecondProducerWhileFirstMakesRoom}}: {{browseLimit=2}}, two exchanges,
then producers A and B with the interleaving above (a delegating list holds A
when it asks for the sub list, until B has completed), then a third exchange C
and a read. On main: {{failures: [producer A:
java.util.ConcurrentModificationException, producer C, after A and B:
java.util.ConcurrentModificationException, browsing, after A, B and C:
java.util.ConcurrentModificationException]}}.
* {{testManyProducerThreads}}: 4 threads, 250 exchanges each,
{{browseLimit=10}}. Depends on scheduling; on main all 4 threads failed with
the exception in each of the 3 attempts of the test run.
* {{testReaderDoesNotSeeTheEndpointEmptyWithLimitOne}}: {{browseLimit=1}}, the
browse status is read while the producer is inside the list. On main: {{sizes
seen by a reader while exchanges arrive: [1, 0, 1, 0]}}.
h3. Proposed fix
Add the exchange, then remove the oldest ones from the list itself:
{{exchanges.subList(0, excess).clear()}} with {{excess = size - browseLimit}}
(one array copy for a {{CopyOnWriteArrayList}}). Both steps are done under a
lock, so that concurrent producers keep exactly {{browseLimit}} exchanges; the
lock is only taken when {{browseLimit}} is set. The filter is not touched.
What a browser sees does not change: the last {{browseLimit}} exchanges in
arrival order ({{BrowseLimitTest}} is unchanged and passes). Because the add
comes first, a reader never sees fewer than the limit once it was reached;
while an exchange arrives it can see {{browseLimit + 1}} for a moment.
{{getExchanges()}} now always returns the endpoint's list, as it already does
without a limit and before the limit is reached.
No upgrade note is needed: no option, default or result changes; a route that
worked keeps working and a browser sees the same exchanges. What goes away is
the retained memory and the exception.
Tests with the fix: {{BrowseLimitMemoryTest}} 5 of 5 pass; the full camel-core
suite: 8171 run, 0 failures, 0 errors, 45 skipped; the browse tests of
camel-management: 19 run, 0 failures.
How it was found: a boundary sweep with a small Lean 4 model of the window
(main: backing list plus offset of the view; fix: a list). The model proves
that main keeps every exchange in the backing list for any limit (so 1000 for
the example above), that the fixed list never has more than {{browseLimit}}
entries, and that the fix shows a browser exactly what main showed. The model
does not cover the {{ConcurrentModificationException}}, concurrent producers or
the moment between add and remove; those are in the Java tests.
Affected: every release since 4.9.0, where CAMEL-21183 added the option (the
4.9.0 tag has the line, 4.8.0 does not). main (a7169d7c6ab6; tested);
camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x have the same line
147 (read, not run).
Duplicate check (2026-10-13): JIRA text "browseLimit" (2: CAMEL-24076 and
CAMEL-24078, the browseLimit option of camel-jms / camel-amqp, other code),
"BrowseEndpoint subList" (none), "browse endpoint memory leak" (5, none about
the browse component); GitHub pull requests "browseLimit": #24735 (camel-jms),
no other.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
