Parker Mauney created CAMEL-26893:
-------------------------------------
Summary: camel-core - adding routes with explicit ids slows
quadratically
Key: CAMEL-26893
URL: https://issues.apache.org/jira/browse/CAMEL-26893
Project: Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 4.23.0, 4.21.0
Reporter: Parker Mauney
When a route definition with an id is added,
{{DefaultModel.removeRouteDefinition}} first calls {{stopRoute(id)}} and then
{{removeRoute(id)}} on the context, so that an existing route with the same id
gets replaced.
{{AbstractCamelContext.removeRoute(String, LoggingLevel)}} builds the
{{endpointsInUse}} map by calling {{gatherEndpoints()}} on every
{{RouteService}} in the context. Only after that does it look up the route
being removed:
{code:java}
lock.lock();
try {
DefaultRouteError.reset(this, routeId);
Map<String, Set<Endpoint>> endpointsInUse = new HashMap<>();
for (Map.Entry<String, RouteService> entry : routeServices.entrySet()) {
endpointsInUse.put(entry.getKey(), entry.getValue().gatherEndpoints());
}
RouteService routeService = routeServices.get(routeId);
if (routeService != null) {
...
}
return false;
} finally {
lock.unlock();
}
{code}
When the id is new, the map is built and then discarded. The same happens when
the route exists but is still started. {{gatherEndpoints()}} walks the child
services of each route, so every added route pays a full walk of every route
already in the context. Adding n routes with explicit ids is therefore O(n²) in
that walk alone.
Routes without an id don't hit this, because {{removeRouteDefinition}} only
calls {{removeRoute}} when the id is set. Applications that add many routes
with explicit ids after the context has started pay that walk for each route.
*Reproducer:* the manual test below adds n small routes, one at a time, to a
started {{DefaultCamelContext}} with JMX disabled, using a fresh context for
each n.
{code:java}
@Disabled("Manual test")
public class AddRoutesWithIdsManualTest {
private static final Logger LOG =
LoggerFactory.getLogger(AddRoutesWithIdsManualTest.class);
@Test
public void testAddRoutesWithIds() throws Exception {
// warm up
addRoutes(1000);
for (int count = 1000; count <= 5000; count += 1000) {
long millis = addRoutes(count);
LOG.info("Added {} routes in {} ms ({} us per route)", count,
millis, millis * 1000 / count);
}
}
private static long addRoutes(int count) throws Exception {
try (DefaultCamelContext context = new DefaultCamelContext()) {
context.disableJMX();
context.start();
long start = System.nanoTime();
for (int i = 0; i < count; i++) {
final int n = i;
context.addRoutes(new RouteBuilder() {
@Override
public void configure() {
from("direct:in" + n).routeId("route" + n)
.filter(header("foo").isNotNull())
.to("mock:out" + n);
}
});
}
return TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);
}
}
}
{code}
To see how much of the cost is that walk, I tried a local build of main where
{{removeRoute}} looks the route up before gathering the endpoints, so the
gather only runs when the route is going to be removed. Results on main
(4d3b74b1cd0b), JDK 25, one run each, without and with that change:
||Routes||main||with the change||
|1,000|1,087 ms (1,087 us/route)|245 ms (245 us/route)|
|2,000|4,394 ms (2,197 us/route)|564 ms (282 us/route)|
|3,000|9,971 ms (3,323 us/route)|885 ms (295 us/route)|
|4,000|18,998 ms (4,749 us/route)|1,675 ms (418 us/route)|
|5,000|30,537 ms (6,107 us/route)|2,425 ms (485 us/route)|
On main, the cost per route grows in proportion to the number of routes already
loaded.
The closest existing issue I found is CAMEL-19456, which is about a different
cost in the same method (the endpoint key mismatch for {{RAW()}} URIs on
removal) and does not touch this gather.
Happy to open a PR with the change and the test above if it helps to see it in
context.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
