Copilot commented on code in PR #5915:
URL: https://github.com/apache/texera/pull/5915#discussion_r3488732844
##########
amber/src/test/integration/org/apache/texera/amber/engine/e2e/ReconfigurationIntegrationSpec.scala:
##########
@@ -166,8 +166,14 @@ class ReconfigurationIntegrationSpec
): Map[OperatorIdentity, List[Tuple]] =
TestUtils.shouldReconfigure(system, ctx, operators, links, targetOps,
newOpExecInitInfo)
+ private def boundedCsvSource() = {
+ val src = TestOperators.mediumCsvScanOpDesc()
+ src.limit = Some(10000)
+ src
+ }
Review Comment:
`boundedCsvSource` calls `TestOperators.mediumCsvScanOpDesc()` but then sets
`src.limit = Some(10000)`. This changes the effective input from the full
medium CSV (used unbounded in `PauseSpec`/`ReconfigurationSpec`) down to 10k
tuples (see `CSVScanSourceOpExec`, which does
`tupleIterator.take(desc.limit.get)`), which conflicts with the PR/issue
rationale of using the full medium source to ensure the workflow stays running
long enough for `pauseWorkflow` to land. Consider removing the limit (or
setting it to match the intended 100k) to avoid reintroducing the
pause-vs-completion race on fast runs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
