eugenegujing opened a new pull request, #7348:
URL: https://github.com/apache/texera/pull/7348
### What changes were proposed in this PR?
`FileScanUtils.createTuplesFromFile` computed the end of its line slice as
`offset + limit.getOrElse(Int.MaxValue)`. With Offset >= 1 and Limit left
empty, the addition overflows `Int` to a negative bound, and `Iterator.slice`
clamps a negative bound to 0 and returns an empty iterator. The File Scan
operator therefore emitted **zero rows, silently, with the workflow reporting
success**. Both `FileScan` and `FileScanOp` delegate to this helper, so both
were affected.
```
Before: offset=1, limit=empty -> slice(1, 1 + Int.MaxValue) -> slice(1,
-2147483648) -> 0 rows
After: offset=1, limit=empty -> drop(1)
-> every remaining row
```
**Offset = 1, Limit left empty — zero rows ("Empty result set") while the
run reports success:**
<img width="1344" height="869" alt="Screenshot 2026-08-05 at 2 19 40 PM"
src="https://github.com/user-attachments/assets/28c628b5-2a71-47a4-bf78-f790c9b113c0";
/>
**Control: Offset = 0, Limit left empty, same file — all five rows:**
<img width="1340" height="869" alt="Screenshot 2026-08-05 at 2 19 47 PM"
src="https://github.com/user-attachments/assets/cd747eee-b931-4e31-8399-ad666881e1cf";
/>
The fix replaces the slice arithmetic with `drop(offset)` plus an optional
`take(limit)` — the shape `CSVScanSourceOpExec` and `ArrowSourceOpExec` already
use — so "no limit" is expressed by not bounding the iterator rather than by a
sentinel value that arithmetic can overflow. Offset and limit still apply per
extracted zip entry (unchanged behavior, now pinned by a test).
### Any related issues, documentation, discussions?
Closes #7345
Same bug class as #7245 (JSONL scan, fixed by #7247).
### How was this PR tested?
TDD: the regression tests were written first and confirmed to fail on the
unfixed code — the four offset-without-limit cases all produced empty output
(e.g. `List() did not equal List("l2", "l3", "l4", "l5")`) — then the fix was
applied and all tests pass.
Ten new cases were added across the three File Scan specs: eight in
`FileScanUtilsSpec` (offset without limit — the regression, offset 0, offset
with limit, limit only, offset past EOF, offset with an `Int.MaxValue` limit,
per-zip-entry offset with `extract = true`, and `isSingle` types ignoring
offset/limit — documented behavior, pinned), plus one operator-level
offset-without-limit case each in `FileScanSourceOpDescSpec` (source operator)
and `FileScanOpDescSpec` (input-port operator), since both operators delegate
to the same helper.
TDD: the regression tests were written first and confirmed to fail on the
unfixed code — the four offset-without-limit cases all produced empty output
(e.g. `List() did not equal List("l2", "l3", "l4", "l5")`) — then the fix was
applied and all tests pass.
Ten new cases were added across the three File Scan specs: eight in
`FileScanUtilsSpec` (offset without limit — the regression, offset 0, offset
with limit, limit only, offset past EOF, offset with an `Int.MaxValue` limit,
per-zip-entry offset with `extract = true`, and `isSingle` types ignoring
offset/limit — documented behavior, pinned), plus one operator-level
offset-without-limit case each in `FileScanSourceOpDescSpec` (source operator)
and `FileScanOpDescSpec` (input-port operator), since both operators delegate
to the same helper.
```bash
sbt "WorkflowOperator/testOnly
org.apache.texera.amber.operator.source.scan.file.FileScanUtilsSpec
org.apache.texera.amber.operator.source.scan.file.FileScanSourceOpDescSpec
org.apache.texera.amber.operator.source.scan.file.FileScanOpDescSpec"
# 29 tests, all passed (19 pre-existing + 10 new)
sbt "WorkflowOperator/scalafixAll --check"
# passed, no lint issues
sbt "WorkflowOperator/scalafmtCheck" "WorkflowOperator/Test/scalafmtCheck"
# passed, no mis-formatted files
sbt WorkflowOperator/test
# full module: 2050 tests in 283 suites, all passed
```
Also verified manually in the UI with the same two-operator workflow shown
in the screenshots above:
<img width="1133" height="762" alt="Screenshot 2026-08-05 at 5 20 41 PM"
src="https://github.com/user-attachments/assets/fe2e3098-ddf4-4787-a611-fcbb3b61e19d";
/>
### Was this PR authored or co-authored using generative AI tooling?
Co-authored by: Claude Code (Claude Fable 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
