This is an automated email from the ASF dual-hosted git repository.
924060929 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 434e8563aae [fix](fe) Allocate a fresh StatementContext per EXECUTE to
prevent FE OOM in long-lived prepared statements (#67256)
434e8563aae is described below
commit 434e8563aaed8dfe45fea646aad2612021745a07
Author: starocean999 <[email protected]>
AuthorDate: Fri Sep 11 16:05:19 2026 +0800
[fix](fe) Allocate a fresh StatementContext per EXECUTE to prevent FE OOM
in long-lived prepared statements (#67256)
Problem Summary:
A prepared statement lives as long as its connection. The
`PreparedStatementContext` kept in
`ConnectContext.preparedStatementContextMap` retains a single
`StatementContext` and reuses the same object across every `EXECUTE` for
the whole connection lifetime.
Because one object is reused across executions, its per-statement state
keeps accumulating: bound tables (`tables`, `oneLevelTables`,
`mtmvRelatedTables`, `insertTargetTables`, `viewInfos`), CTE maps,
statistics (`relationIdToStatisticsMap`, `tableIdMapping`), MV/partition
rewrite state (`mvCanRewritePartitionsMap`, `tmpPlanForMvRewrite`,
`materializationRewrittenSuccessSet`), MVCC snapshots, connector write
schemas, placeholder bindings (`idToPlaceholderRealExpr`), etc. On
long-lived connections with a high number of `EXECUTE`s, these maps only
grow and are never released until the connection closes, which can OOM
the FE.
**Root cause:** the `StatementContext` stored in
`PreparedStatementContext` was treated as a permanent
per-prepared-statement object and reused, so state that should be
per-execution lived as long as the connection.
**Fix:** instead of reusing (and clearing in place) the same
`StatementContext`, allocate a brand-new context on every `EXECUTE` and
carry over only the state that must survive between executions:
- **ID generator positions** — so ids generated during this execution
never collide with ids already present in the cached analyzed plan from
`PREPARE`;
- **placeholder real expressions** bound by the protocol layer for this
`EXECUTE` (`idToPlaceholderRealExpr`) — this is the piece that prevents
the #63920 parameter-mismatch regression;
- the **placeholder → comparison-slot registry** (`idToComparisonSlot`)
used by the short-circuit fast path;
- the **placeholder list**;
- the **short-circuit / nondeterministic flags** that gate the
short-circuit fast path before any re-planning.
After the swap, the previous context becomes unreachable and is promptly
GC'd, so memory no longer grows with the number of executions. The
cached analyzed plan and the point-query (short-circuit) cache live on
`PrepareCommand` and `PreparedStatementContext` respectively, so they
keep being reused across executions.
**Changes:**
- `IdGenerator`: add `getCurrentId()` so a fresh context can continue
the id generators from the previous one.
- `StatementContext`: add `createNextExecuteContext()` which allocates
the fresh context and copies over the cross-execution state above.
- `PreparedStatementContext`: add `nextStatementContext()` which swaps
in the fresh context so the old one is released.
- `ExecuteCommand`: `run()` now uses the fresh per-execution context
(and the now-redundant in-place
`resetConnectorStatementScope()`/`resetMvccSnapshots()` calls are
removed since a fresh context starts empty by construction).
- Unit tests updated to assert the fresh-context behavior
(`ExecuteCommandTest`, `ConnectorStatementScopeTest`).
None
---
.../java/org/apache/doris/common/IdGenerator.java | 8 ++
.../org/apache/doris/nereids/StatementContext.java | 47 ++++++++++
.../trees/plans/commands/ExecuteCommand.java | 17 ++--
.../apache/doris/qe/PreparedStatementContext.java | 24 ++++++
.../connector/ConnectorStatementScopeTest.java | 51 +++++++----
.../trees/plans/commands/ExecuteCommandTest.java | 99 +++++++++++++++++++---
6 files changed, 213 insertions(+), 33 deletions(-)
diff --git
a/fe/fe-common/src/main/java/org/apache/doris/common/IdGenerator.java
b/fe/fe-common/src/main/java/org/apache/doris/common/IdGenerator.java
index 76f503bb5d7..aa769cf3415 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/IdGenerator.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/IdGenerator.java
@@ -33,6 +33,14 @@ public abstract class IdGenerator<IdType extends Id<IdType>>
{
return this;
}
+ /**
+ * The id value that {@link #getNextId()} would hand out next. Used to
seed a fresh
+ * per-execution id generator from an existing one so ids never collide.
+ */
+ public int getCurrentId() {
+ return nextId;
+ }
+
public abstract IdType getNextId();
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/StatementContext.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/StatementContext.java
index 0ea2e7cb025..64670a7398b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/StatementContext.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/StatementContext.java
@@ -400,6 +400,53 @@ public class StatementContext implements Closeable {
}
}
+ /**
+ * Create a fresh StatementContext for the next EXECUTE of a prepared
statement.
+ *
+ * <p>A prepared statement keeps its StatementContext inside {@code
PreparedStatementContext}
+ * for the whole lifetime of the connection. Reusing the same object
across executions makes
+ * its per-statement state (bound tables, CTE maps, statistics, snapshots,
connector scope,
+ * ...) accumulate and it is only released when the connection closes,
which can OOM
+ * long-lived connections. Instead of clearing in place, allocate a
brand-new context per
+ * EXECUTE and copy over only the state that must survive between
executions, so the previous
+ * context becomes unreachable and is promptly GC'd.
+ *
+ * <p>Carried over:
+ * <ul>
+ * <li>id generator positions, so ids generated during this execution
never collide with
+ * ids already present in the cached analyzed plan from PREPARE;</li>
+ * <li>the placeholder real expressions bound by this EXECUTE (the
protocol layer fills
+ * them on the previous context before this method runs) and the
placeholder list;</li>
+ * <li>the placeholder to comparison-slot registry used by the
short-circuit fast path;</li>
+ * <li>the short-circuit and nondeterministic flags that gate the
short-circuit fast path
+ * before this execution re-plans.</li>
+ * </ul>
+ * Everything else (tables, CTEs, statistics, snapshots, planner
resources, connector
+ * scope, ...) starts empty/fresh on the new context.
+ */
+ public StatementContext createNextExecuteContext() {
+ // Continue the id generators from the previous context. The cached
analyzed plan from
+ // PREPARE (and every prior execution) already consumed ids from them,
so a fresh
+ // generator starting at 0 would collide with those ids during this
execution's planning.
+ StatementContext next = new StatementContext(connectContext,
originStatement,
+ exprIdGenerator.getCurrentId());
+ next.objectIdGenerator.resetId(objectIdGenerator.getCurrentId());
+ next.relationIdGenerator.resetId(relationIdGenerator.getCurrentId());
+ next.cteIdGenerator.resetId(cteIdGenerator.getCurrentId());
+ next.talbeIdGenerator.resetId(talbeIdGenerator.getCurrentId());
+
next.placeHolderIdGenerator.resetId(placeHolderIdGenerator.getCurrentId());
+ // Placeholder bindings of this EXECUTE, and the comparison-slot
registry used to replace
+ // conjuncts on the cached short-circuit plan without re-planning.
+ next.idToPlaceholderRealExpr.putAll(idToPlaceholderRealExpr);
+ next.idToComparisonSlot.putAll(idToComparisonSlot);
+ next.placeholders = new ArrayList<>(placeholders);
+ // Short-circuit gating flags are computed by the previous execution's
planning and gate
+ // the fast path of this execution before any re-planning happens.
+ next.isShortCircuitQuery = isShortCircuitQuery;
+ next.hasNondeterministic = hasNondeterministic;
+ return next;
+ }
+
public void setNeedLockTables(boolean needLockTables) {
this.needLockTables = needLockTables;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java
index dd7b551585c..ee09d2a7d1a 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java
@@ -90,14 +90,14 @@ public class ExecuteCommand extends Command {
"prepare statement " + stmtName + " not found, maybe
expired");
}
PrepareCommand prepareCommand = preparedStmtCtx.command;
- StatementContext statementContext =
preparedStmtCtx.getStatementContext();
+ // Allocate a fresh StatementContext per EXECUTE so the per-statement
state accumulated by
+ // prior executions (bound tables, CTE maps, statistics, snapshots,
...) is released
+ // promptly instead of living as long as the connection, which can OOM
long-lived
+ // connections. The necessary cross-execution state (placeholder
bindings, comparison
+ // slots, id generator positions, short-circuit flags) is carried over
to the new context.
+ StatementContext statementContext =
preparedStmtCtx.nextStatementContext();
statementContext.setPrepareStage(false);
statementContext.setIsInsert(false);
- // A prepared EXECUTE reuses this one StatementContext across
executions; drop the connector
- // per-statement scope so a prior execution's cached tables/state
never leak into this one (the
- // scope key's queryId is a second line of defense). See
StatementContext#resetConnectorStatementScope.
- statementContext.resetConnectorStatementScope();
- statementContext.resetMvccSnapshots();
LogicalPlan logicalPlan = prepareCommand.getLogicalPlan();
List<LogicalPlan> relationRoots = new ArrayList<>();
if (logicalPlan instanceof InsertIntoTableCommand) {
@@ -157,6 +157,11 @@ public class ExecuteCommand extends Command {
&& hasShortCircuitContext
&& shortCircuitContextReusable
&& !statementContext.hasNondeterministic()) {
+ // The fresh per-execution context carries the short-circuit flag
but not the cached plan.
+ // Install the just-validated cache before the direct path: result
sending reads it via
+ // statementContext.getShortCircuitQueryContext(), and the
fallback (building one from a
+ // null planner, since this path skips planning) would NPE.
+
statementContext.setShortCircuitQueryContext(preparedStmtCtx.shortCircuitQueryContext.get());
PointQueryExecutor.directExecuteShortCircuitQuery(executor,
preparedStmtCtx, statementContext);
return;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/qe/PreparedStatementContext.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/PreparedStatementContext.java
index fd1b093ee12..f28f6bd1c53 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/PreparedStatementContext.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/PreparedStatementContext.java
@@ -54,6 +54,30 @@ public class PreparedStatementContext {
this.statementContext = statementContext;
}
+ /**
+ * Allocate a fresh StatementContext for this EXECUTE and replace the
previous one, so the
+ * old context (with the per-statement state accumulated by prior
executions: bound tables,
+ * CTE maps, statistics, snapshots, ...) becomes unreachable and is
promptly GC'd.
+ *
+ * <p>A prepared statement lives as long as its connection. Reusing one
StatementContext
+ * across all executions would keep growing those maps and could OOM
long-lived connections,
+ * so we create a new object per execution and carry over only the state
that must survive
+ * (placeholder bindings, comparison slots, id generator positions,
short-circuit flags).
+ *
+ * @return the fresh StatementContext to use for the current execution
+ */
+ public StatementContext nextStatementContext() {
+ // Close the outgoing context's per-statement connector scope before
dropping it. The binary
+ // COM_STMT_EXECUTE path has no per-statement StatementContext.close()
finally (that only
+ // runs for COM_QUERY), and coordinated scans may not have registered
a query-finish
+ // callback yet (connector commands and failures before scan
registration have none).
+ // Without this, the outgoing scope's closeable connector metadata /
active connector
+ // transactions would be abandoned, and GC cannot finalize them.
+ statementContext.resetConnectorStatementScope();
+ statementContext = statementContext.createNextExecuteContext();
+ return statementContext;
+ }
+
public void setStartTime() {
startTime = System.currentTimeMillis();
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/connector/ConnectorStatementScopeTest.java
b/fe/fe-core/src/test/java/org/apache/doris/connector/ConnectorStatementScopeTest.java
index 32a38415658..428ac2a2b80 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/connector/ConnectorStatementScopeTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/connector/ConnectorStatementScopeTest.java
@@ -45,9 +45,10 @@ import java.util.concurrent.atomic.AtomicInteger;
/**
* Tests for the per-statement {@link ConnectorStatementScope}: the {@link
ConnectorStatementScope#NONE} no-op,
* the memoizing {@link ConnectorStatementScopeImpl}, the {@link
StatementContext} hosting + per-execution
- * reset a reused prepared statement relies on, and that {@link
ExecuteCommand} actually invokes that reset on
- * every execution (external/connector tables are planned through the reused
prepared context, so a missing
- * reset would leak one execution's loaded table into the next — see {@code
executeCommandResetsConnectorScope*}).
+ * reset primitive, and that {@link ExecuteCommand} allocates a fresh
StatementContext (and therefore a fresh
+ * connector scope) on every execution (external/connector tables are planned
through the context, so a missing
+ * fresh context would leak one execution's loaded table into the next — see
+ * {@code executeCommandAllocatesFreshConnectorScopePerExecution}).
*/
public class ConnectorStatementScopeTest {
@@ -184,21 +185,27 @@ public class ConnectorStatementScopeTest {
}
@Test
- public void executeCommandResetsConnectorScopePerExecution() throws
Exception {
- // WIRING test: the reset above is only load-bearing if
ExecuteCommand.run() actually calls it. A prepared
- // statement reuses ONE StatementContext across every EXECUTE, and an
EXTERNAL/connector table is planned
- // through that reused context each execution (external tables never
take the OLAP short-circuit fast path;
- // they always fall to the normal executor.execute() planner, which
re-resolves the table per execution).
- // So run() must drop the connector per-statement scope at the top of
every execution, or a prior execution's
- // memoized (loaded) table leaks into the next. The tests above cover
only the reset PRIMITIVE; this covers
- // that the command invokes it. MUTATION: delete
`statementContext.resetConnectorStatementScope()` from
- // ExecuteCommand.run() -> the seeded value survives -> the
assertNotSame below flips -> red.
+ public void executeCommandAllocatesFreshConnectorScopePerExecution()
throws Exception {
+ // WIRING test: a prepared statement lives as long as its connection,
so ExecuteCommand must not let one
+ // execution's connector state leak into the next. Previously run()
reused ONE StatementContext and reset
+ // its connector per-statement scope at the top of every execution;
now it allocates a brand-new
+ // StatementContext per EXECUTE (dropping the previous one, whose
per-statement maps were the OOM source
+ // on long-lived connections), so the fresh context starts with a
brand-new connector scope by
+ // construction. External/connector tables never take the OLAP
short-circuit fast path; they always fall
+ // to the normal executor.execute() planner, which re-resolves the
table per execution through that fresh
+ // context. MUTATION: reverting to reusing one StatementContext across
executions (without a reset) -> the
+ // seeded value survives in the context still used by the next
execution -> the assertNotSame below flips
+ // -> red.
StatementContext statementContext = new StatementContext();
- // Seed a value the way a first execution's connector planning would
(one loaded table the statement shares).
+ // Seed values the way a first execution's connector planning would
(one loaded table the statement
+ // shares, plus closeable metadata that must be finalized when the
context is dropped).
ConnectorStatementScope firstScope =
statementContext.getOrCreateConnectorStatementScope();
Object memoizedTable = firstScope.computeIfAbsent("table:1",
Object::new);
+ AtomicInteger closes = new AtomicInteger();
+ AutoCloseable closeable = () -> closes.incrementAndGet();
+ firstScope.computeIfAbsent("closeable:1", () -> closeable);
- // A prepared statement wrapping that reused StatementContext, with a
plain (non-cache, non-insert) plan.
+ // A prepared statement wrapping that StatementContext, with a plain
(non-cache, non-insert) plan.
LogicalPlan plan = Mockito.mock(LogicalPlan.class);
PrepareCommand prepareCommand = Mockito.mock(PrepareCommand.class);
Mockito.when(prepareCommand.getLogicalPlan()).thenReturn(plan);
@@ -213,15 +220,25 @@ public class ConnectorStatementScopeTest {
Mockito.when(ctx.getSessionVariable()).thenReturn(sessionVariable);
Mockito.when(ctx.getStatementContext()).thenReturn(statementContext);
- // A no-op executor: we pin the reset wiring, not the planner.
execute() is a mock no-op.
+ // A no-op executor: we pin the fresh-context wiring, not the planner.
execute() is a mock no-op.
StmtExecutor executor = Mockito.mock(StmtExecutor.class);
Mockito.when(executor.getContext()).thenReturn(ctx);
new ExecuteCommand("s", prepareCommand, statementContext).run(ctx,
executor);
- ConnectorStatementScope secondScope =
statementContext.getOrCreateConnectorStatementScope();
+ // run() must swap in a fresh StatementContext for this execution,
releasing the previous one...
+ StatementContext nextContext = preparedStmtCtx.getStatementContext();
+ Assertions.assertNotSame(statementContext, nextContext,
+ "ExecuteCommand allocates a fresh StatementContext per EXECUTE
so the old one is released");
+ // ...finalizing the outgoing scope's closeable connector values
(COM_STMT_EXECUTE has no per-statement
+ // StatementContext.close() finally, so dropping the context must
close the scope, or closeable metadata
+ // and active connector transactions would be abandoned and never
finalized by GC).
+ Assertions.assertEquals(1, closes.get(),
+ "dropping the outgoing context finalizes its closeable
connector values");
+ // ...and whose connector scope is brand-new, so a prior execution's
memoized connector table cannot leak.
+ ConnectorStatementScope secondScope =
nextContext.getOrCreateConnectorStatementScope();
Assertions.assertNotSame(firstScope, secondScope,
- "ExecuteCommand drops the reused context's connector scope so
the next execution starts fresh");
+ "the fresh context starts with a fresh connector scope");
Assertions.assertNotSame(memoizedTable,
secondScope.computeIfAbsent("table:1", Object::new),
"a prior execution's memoized connector table must not leak
into the next EXECUTE");
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommandTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommandTest.java
index 8872c6e7b65..7905b31b5ef 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommandTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommandTest.java
@@ -17,9 +17,12 @@
package org.apache.doris.nereids.trees.plans.commands;
+import org.apache.doris.analysis.DescriptorTable;
+import org.apache.doris.analysis.Queriable;
import org.apache.doris.analysis.TableScanParams;
import org.apache.doris.catalog.Column;
import org.apache.doris.catalog.DatabaseIf;
+import org.apache.doris.catalog.OlapTable;
import org.apache.doris.catalog.PrimitiveType;
import org.apache.doris.catalog.ScalarType;
import org.apache.doris.catalog.TableIf;
@@ -32,11 +35,15 @@ import org.apache.doris.nereids.parser.NereidsParser;
import org.apache.doris.nereids.trees.expressions.SubqueryExpr;
import org.apache.doris.nereids.trees.plans.commands.merge.MergeIntoCommand;
import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
+import org.apache.doris.planner.OlapScanNode;
+import org.apache.doris.planner.Planner;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.qe.OriginStatement;
import org.apache.doris.qe.PreparedStatementContext;
import org.apache.doris.qe.SessionVariable;
+import org.apache.doris.qe.ShortCircuitQueryContext;
import org.apache.doris.qe.StmtExecutor;
+import org.apache.doris.thrift.TQueryOptions;
import com.google.common.collect.ImmutableMap;
import org.junit.jupiter.api.Assertions;
@@ -141,11 +148,11 @@ public class ExecuteCommandTest {
@Test
public void
testPreparedConnectorUpdateRefreshesWriteDefaultEveryExecution() throws
Exception {
- // Prepared UPDATE reuses one StatementContext. Model connector
metadata changing from default 1 to 2
- // between executions: each planner callback pins the current schema
only when no schema is already pinned,
- // then expands DEFAULT(v) and writes the resulting value.
- // MUTATION: resetConnectorStatementScope() not clearing
connectorWriteSchemas makes execution two reuse
- // default 1, so the written values become [1, 1] instead of [1, 2].
+ // ExecuteCommand allocates a fresh StatementContext per EXECUTE.
Model connector metadata changing from
+ // default 1 to 2 between executions: each execution's planner
callback pins the current schema only when
+ // the fresh context has no schema pinned, then expands DEFAULT(v) and
writes the resulting value.
+ // MUTATION: reusing one StatementContext across executions without
dropping connectorWriteSchemas makes
+ // execution two reuse default 1, so the written values become [1, 1]
instead of [1, 2].
String sql = "update ext_catalog.db.t set v = default(v) where id = 1";
LogicalPlan logicalPlan = new NereidsParser().parseSingle(sql);
Assertions.assertInstanceOf(UpdateCommand.class, logicalPlan);
@@ -166,12 +173,16 @@ public class ExecuteCommandTest {
AtomicInteger metadataDefault = new AtomicInteger(1);
List<String> writtenValues = new ArrayList<>();
Mockito.doAnswer(invocation -> {
- if
(!statementContext.getConnectorWriteSchema(tableId).isPresent()) {
+ // Each execution plans through the fresh StatementContext
allocated by ExecuteCommand, so resolve/pin
+ // the connector writer schema on THAT context (a stale pin would
make the second execution reuse
+ // default 1).
+ StatementContext currentContext =
preparedStatement.getStatementContext();
+ if (!currentContext.getConnectorWriteSchema(tableId).isPresent()) {
Column column = new Column("v",
ScalarType.createType(PrimitiveType.INT),
false, null, String.valueOf(metadataDefault.get()),
"");
- statementContext.setConnectorWriteSchema(tableId,
Collections.singletonList(column));
+ currentContext.setConnectorWriteSchema(tableId,
Collections.singletonList(column));
}
-
writtenValues.add(statementContext.getConnectorWriteSchema(tableId).get()
+
writtenValues.add(currentContext.getConnectorWriteSchema(tableId).get()
.get(0).getDefaultValueSql());
return null;
}).when(executor).execute();
@@ -219,13 +230,81 @@ public class ExecuteCommandTest {
statementContext.getSnapshot(table, Optional.empty(),
Optional.empty()).orElse(null));
new ExecuteCommand("stmt", prepareCommand,
statementContext).run(connectContext, executor);
- statementContext.loadSnapshots(table, Optional.empty(),
Optional.empty());
+
+ // ExecuteCommand allocates a fresh StatementContext per EXECUTE, so
the next execution must not reuse the
+ // snapshot pinned on the previous context (a stale snapshot would
make a later commit permanently
+ // invisible).
+ StatementContext nextContext = preparedStatement.getStatementContext();
+ Assertions.assertNotSame(statementContext, nextContext,
+ "ExecuteCommand allocates a fresh StatementContext per
EXECUTE");
+ nextContext.loadSnapshots(table, Optional.empty(), Optional.empty());
Assertions.assertSame(second,
- statementContext.getSnapshot(table, Optional.empty(),
Optional.empty()).orElse(null));
+ nextContext.getSnapshot(table, Optional.empty(),
Optional.empty()).orElse(null));
Mockito.verify(table, Mockito.times(2)).loadSnapshot(Optional.empty(),
Optional.empty());
}
+ @Test
+ public void
testFastPathInstallsCachedShortCircuitContextAcrossExecutions() throws
Exception {
+ // ExecuteCommand allocates a fresh StatementContext per EXECUTE. The
fresh context carries the
+ // short-circuit flag but not the cached plan, so the fast path must
install the just-validated
+ // ShortCircuitQueryContext before direct execution -- otherwise
result sending falls back to
+ // `new ShortCircuitQueryContext(planner, ...)` with a null planner
(this path never plans) and
+ // NPEs on planner.getDescTable(). Two executions exercise the second
(reusable) EXECUTE that
+ // hits the regression.
+ // MUTATION: removing the install in ExecuteCommand.run() -> the fresh
context has no
+ // statement-level cache -> the assertSame below flips -> red.
+ String sql = "select * from tbl";
+ LogicalPlan logicalPlan = new NereidsParser().parseSingle(sql);
+
+ ConnectContext connectContext = Mockito.mock(ConnectContext.class);
+ StatementContext statementContext = new StatementContext();
+ statementContext.setShortCircuitQuery(true);
+ PrepareCommand prepareCommand = new PrepareCommand(
+ "stmt", logicalPlan, Collections.emptyList(), new
OriginStatement(sql, 0));
+ PreparedStatementContext preparedStatement = new
PreparedStatementContext(
+ prepareCommand, connectContext, statementContext, "stmt");
+
+ // A real ShortCircuitQueryContext (built from a mocked planner) that
passes isReusable().
+ Planner planner = Mockito.mock(Planner.class);
+ Mockito.when(planner.getQueryOptions()).thenReturn(new
TQueryOptions());
+ DescriptorTable descriptorTable = new DescriptorTable();
+ descriptorTable.createTupleDescriptor();
+ Mockito.when(planner.getDescTable()).thenReturn(descriptorTable);
+ OlapScanNode scanNode = Mockito.mock(OlapScanNode.class);
+ OlapTable table = Mockito.spy(new OlapTable());
+ Mockito.doReturn("tbl").when(table).getName();
+ Mockito.doReturn(10).when(table).getBaseSchemaVersion();
+
Mockito.when(scanNode.getPointQueryProjectList()).thenReturn(Collections.emptyList());
+ Mockito.when(scanNode.getOlapTable()).thenReturn(table);
+ Mockito.when(scanNode.getTableNameInPlan()).thenReturn("tbl");
+
Mockito.when(scanNode.getConjuncts()).thenReturn(Collections.emptyList());
+
Mockito.when(planner.getScanNodes()).thenReturn(Collections.singletonList(scanNode));
+ ShortCircuitQueryContext cachedPlan = new
ShortCircuitQueryContext(planner, Mockito.mock(Queriable.class));
+ preparedStatement.shortCircuitQueryContext = Optional.of(cachedPlan);
+
+ StmtExecutor executor = Mockito.mock(StmtExecutor.class);
+
Mockito.when(connectContext.getPreparedStementContext("stmt")).thenReturn(preparedStatement);
+ SessionVariable sessionVariable = new SessionVariable();
+ sessionVariable.enableGroupCommitFullPrepare = false;
+
Mockito.when(connectContext.getSessionVariable()).thenReturn(sessionVariable);
+
Mockito.when(connectContext.getStatementContext()).thenReturn(statementContext);
+ Mockito.when(executor.getContext()).thenReturn(connectContext);
+
+ ExecuteCommand execute = new ExecuteCommand("stmt", prepareCommand,
statementContext);
+ execute.run(connectContext, executor);
+ Assertions.assertSame(cachedPlan,
preparedStatement.getStatementContext().getShortCircuitQueryContext(),
+ "the fast path installs the validated cache on the fresh
context (first EXECUTE)");
+ Mockito.verify(executor,
Mockito.times(1)).executeAndSendResult(Mockito.anyBoolean(),
Mockito.anyBoolean(),
+ Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any());
+
+ execute.run(connectContext, executor);
+ Assertions.assertSame(cachedPlan,
preparedStatement.getStatementContext().getShortCircuitQueryContext(),
+ "the fast path installs the validated cache on the fresh
context (second, reusable EXECUTE)");
+ Mockito.verify(executor,
Mockito.times(2)).executeAndSendResult(Mockito.anyBoolean(),
Mockito.anyBoolean(),
+ Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any());
+ }
+
private String resolveNextSnapshot(TableScanParams scanParams,
AtomicInteger snapshotId) {
return scanParams.getOrResolveMapParams(ignored -> ImmutableMap.of(
"scan.snapshot-id",
String.valueOf(snapshotId.incrementAndGet())))
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
