Arvind Kandpal created NIFI-16600:
-------------------------------------
Summary: CSVReader with FastCSV parser treats first data row as
header when Treat First Line as Header is false
Key: NIFI-16600
URL: https://issues.apache.org/jira/browse/NIFI-16600
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Arvind Kandpal
When CSVReader uses CSV Parser = FastCSV and "Treat First Line as Header" is
false, the first data row is still read as the header. That row is lost, and
the remaining rows are mapped to the wrong field names.
In the FastCSVRecordReader constructor, the header map is built from the schema
when hasHeader is false. But getRecordFields() always reads the first CSV
record as the header row and replaces that header map, without checking
hasHeader.
Steps to reproduce:
1. Configure CSVReader:
- CSV Parser: FastCSV
- CSV Format: RFC 4180
- Schema Access Strategy: Use 'Schema Text' Property
- Schema Text: record with fields id (int) and name (string)
- Treat First Line as Header: false
2. Read this content (no header line):
1,john
2,jane
Expected: two records, \{id=1, name=john} and \{id=2, name=jane}. This is what
the Apache Commons CSV and Jackson CSV parsers return.
Actual: one record, \{1=2, john=jane}. The first row is used as the header, id
and name are not set, and the first row of data is lost.
Code: nifi-record-serialization-services, FastCSVRecordReader.java,
getRecordFields()
The existing tests in TestFastCSVRecordReader only cover hasHeader = true.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)