github-actions[bot] commented on code in PR #69850:
URL: https://github.com/apache/doris/pull/69850#discussion_r4423124345
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/stream/OlapTableStream.java:
##########
@@ -318,6 +375,7 @@ public void
unprotectedUpdateStreamUpdate(AbstractTableStreamUpdate update, Long
partitionOffset.put(entry.getKey(), entry.getValue());
partitionConsumptionTime.put(entry.getKey(), ts);
}
+ unprotectedRecordPartitionStreamStates(next.keySet());
Review Comment:
[P1] Keep the stream range and row-count baseline from one atomic snapshot.
`OlapTableStreamWrapper` builds `outputUpdateMap` from
`stream.getStreamUpdate()` without the stream lock, then
`estimatePartitionChangelogRead()` reads `getPartitionStreamState()` later. A
concurrent `unprotectedUpdateStreamUpdate()` can advance the offset and replace
the baseline at this line, so the planner pairs the old `(start,end]` range
with the post-advance counter; `changelogDelta()` can then return zero or a
stale low count for a nonempty range while FE row-count reports lag, recreating
the one-row/broadcast-build risk this change is meant to fix. Planning locks
the base `OlapTable`, not the `OlapTableStream`, and after restart the
persisted non-null `Map` bypasses the `ConcurrentHashMap` initializer. Capture
offset and state under one stream read lock or versioned immutable snapshot and
normalize deserialized state to a concurrent map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
