Arvind Kandpal created NIFI-16600:
-------------------------------------

             Summary: CSVReader with FastCSV parser treats first data row as 
header when Treat First Line as Header is false
                 Key: NIFI-16600
                 URL: https://issues.apache.org/jira/browse/NIFI-16600
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
            Reporter: Arvind Kandpal


When CSVReader uses CSV Parser = FastCSV and "Treat First Line as Header" is 
false, the first data row is still read as the header. That row is lost, and 
the remaining rows are mapped to the wrong field names.

In the FastCSVRecordReader constructor, the header map is built from the schema 
when hasHeader is false. But getRecordFields() always reads the first CSV 
record as the header row and replaces that header map, without checking 
hasHeader.

Steps to reproduce:
1. Configure CSVReader:
 - CSV Parser: FastCSV
 - CSV Format: RFC 4180
 - Schema Access Strategy: Use 'Schema Text' Property
 - Schema Text: record with fields id (int) and name (string)
 - Treat First Line as Header: false
2. Read this content (no header line):
1,john
2,jane

Expected: two records, \{id=1, name=john} and \{id=2, name=jane}. This is what 
the Apache Commons CSV and Jackson CSV parsers return.

Actual: one record, \{1=2, john=jane}. The first row is used as the header, id 
and name are not set, and the first row of data is lost.

Code: nifi-record-serialization-services, FastCSVRecordReader.java, 
getRecordFields()

The existing tests in TestFastCSVRecordReader only cover hasHeader = true.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to