kz930 opened a new issue, #8602:
URL: https://github.com/apache/texera/issues/8602
### What happened?
The Limit on a file-scan source is meant to bound how many rows the operator
outputs. It also bounds the sample `sourceSchema()` reads to infer the columns,
so a Limit of 0 leaves the inference nothing to look at and the operator
reports a schema that does not describe the file.
The three readers fail differently on the same file. CSV File Scan and JSONL
File Scan return a schema with no attributes at all. CSVOld File Scan throws
`ArrayIndexOutOfBoundsException`, because it takes the column names from the
header row and the types from the sample, then asks the empty type array for
the first header's type.
Zero is a value the property editor accepts, and it is the natural way to
ask for no rows. The columns of a file do not depend on how many of its rows
the user wants, so a Limit of 0 should give an operator with the file's schema
and an empty output, not one with no columns or one that throws.
### How to reproduce?
Point a CSVOld File Scan at a CSV with a header and a couple of data rows,
set Limit to 0, and run. The execution fails as the executor is constructed, on
the operator's own schema inference, before a row is read. The same two steps
on a CSV File Scan or a JSONL File Scan instead give downstream operators an
operator with no columns.
Set Limit to 1 on any of the three and the schema is correct again, which is
what tells the two concerns apart: `CSVOldScanSourceOpDesc.sourceSchema` reads
its inference sample from `slice(startOffset, startOffset + limit)`, and
`CSVScanSourceOpDesc` and `JSONLScanSourceOpDesc` bound theirs the same way.
### Version/Branch
1.4.0-incubating-SNAPSHOT (main)
### Commit Hash (Optional)
e7d1676e1
### What browsers are you seeing the problem on?
_No response_
### Relevant log output
```shell
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at
org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.$anonfun$sourceSchema$8(CSVOldScanSourceOpDesc.scala:204)
at
org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.sourceSchema(CSVOldScanSourceOpDesc.scala:201)
at
org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpExec.<init>(CSVOldScanSourceOpExec.scala:38)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
