[
https://issues.apache.org/jira/browse/NIFI-4510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-4510.
----------------------------------
Resolution: Feedback Received
Apache NiFi 1.x is no longer maintained and no new release is planned on the
1.x release line. Marking as resolved as part of a cleanup operation. Please
open a new one with an updated description if this is still relevant for NiFi
2.x.
> ValidateRecord does not work properly with AvroRecordSetWriter
> --------------------------------------------------------------
>
> Key: NIFI-4510
> URL: https://issues.apache.org/jira/browse/NIFI-4510
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.4.0
> Environment: Hortonworks HDF Sandbox with inbuilt NiFi 1.2 disabled,
> and NiFi 1.4 downloaded & running
> Reporter: David Doran
> Priority: Major
> Attachments: ValidateRecordTest.xml
>
>
> When using CSVReader and JsonRecordSetWriter, the ValidateRecord processor
> works as expected: Valid records are emitted as a flowfile on the valid
> queue, invalid ones on the invalid queue.
> However, when using CSVReader and AvroRecordSetWriter, the presence of an
> invalid record causes the ValidateRecord processor to fail: Nothing is
> emitted on any of the downstream connectors (failure, invalid or valid).
> Instead the session is rolled back and the input file is left in the upstream
> queue.
> Here's the simple schema I've been using:
> {
> "type": "record",
> "name": "test",
> "fields": [
> {
> "name": "Key",
> "type": "string"
> },
> {
> "name": "ShouldBeLong",
> "type": "long"
> }]
> }
> And here's some sample CSV data:
> TheKey,123
> TheKey,456
> TheKey,NotALong1
> TheKey,NotALong2
> TheKey,NotALong3
> TheKey,321
> TheKey,654
> Using CSVReader->JsonRecordSetWriter results in a flowfile in the valid path:
> [ {
> "Key" : "TheKey",
> "ShouldBeLong" : 123
> }, {
> "Key" : "TheKey",
> "ShouldBeLong" : 456
> }, {
> "Key" : "TheKey",
> "ShouldBeLong" : 321
> }, {
> "Key" : "TheKey",
> "ShouldBeLong" : 654
> } ]
> and in invalid path:
> [ {
> "Key" : "TheKey",
> "ShouldBeLong" : "NotALong1"
> }, {
> "Key" : "TheKey",
> "ShouldBeLong" : "NotALong2"
> }, {
> "Key" : "TheKey",
> "ShouldBeLong" : "NotALong3"
> } ]
> … as expected.
> With CSVReader->AvroRecordSetWriter, the ValidateRecord processor bulletins
> errors repeatedly (because it keeps retrying) and the incoming flow file
> remains in the input queue:
> 22:40:22 UTC ERROR 015f100a-3b6f-1638-43d1-143f4ca4a816
> ValidateRecord[id=015f100a-3b6f-1638-43d1-143f4ca4a816]
> ValidateRecord[id=015f100a-3b6f-1638-43d1-143f4ca4a816] failed to process due
> to java.lang.NumberFormatException: For input string: "NotALong1"; rolling
> back session: For input string: "NotALong1"
>
> 22:40:22 UTC ERROR 015f100a-3b6f-1638-43d1-143f4ca4a816
> ValidateRecord[id=015f100a-3b6f-1638-43d1-143f4ca4a816]
> ValidateRecord[id=015f100a-3b6f-1638-43d1-143f4ca4a816] failed to process
> session due to java.lang.NumberFormatException: For input string:
> "NotALong1": For input string: "NotALong1"
> Thanks,
> Dave.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
