[
https://issues.apache.org/jira/browse/NIFI-14753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dariusz Seweryn updated NIFI-14753:
-----------------------------------
Description:
h1. Context
Consider a WriteJsonResult for schema ["Int Field Name": INT].
Writing a record with ["Int Field Name": Long.MAX_VALUE] throws an
IllegalTypeConversionException — this is expected. Unfortunately the
ByteArrayOutputStream (BAOS) is then left with content that is not a valid JSON:
{ "Int Field Name" }
See [this
discussion|https://github.com/apache/nifi/pull/10053#discussion_r2190950596].
h1. IllegalTypeConversionException handling
This situation has the following consequences for exception handling apart from
handling the offending record in the processor that tried to write the record:
The same malformed record will most probably surface/cause a warning log in a
following processor which can be surprising without the knowledge of this
behavior in WriteJsonResult (unless all other processors assume that Malformed
Records may appear in consumed FlowFiles and silently drop such records).
{_}This can lead to unnecessary work being wasted by people triaging such
warning logs{_}.
h1. Ideal solution
WriteJsonResult in case of IllegalTypeConversionException does not write any
data to the BAOS.
Such approach would need to buffer all writes to the FlowFile until full record
is processed, executing them afterwards. This may impact performance.
was:
h1. Context
Consider a WriteJsonResult for schema ["Int Field Name": INT].
Writing a record with ["Int Field Name": Long.MAX_VALUE] throws an
IllegalTypeConversionException — this is expected. Unfortunately the
ByteArrayOutputStream (BAOS) is then left with content that is not a valid JSON:
{ "Int Field Name" }
h1. IllegalTypeConversionException handling
This situation has the following consequences for exception handling apart from
handling the offending record in the processor that tried to write the record:
The same malformed record will most probably surface/cause a warning log in a
following processor which can be surprising without the knowledge of this
behavior in WriteJsonResult (unless all other processors assume that Malformed
Records may appear in consumed FlowFiles and silently drop such records).
{_}This can lead to unnecessary work being wasted by people triaging such
warning logs{_}.
h1. Ideal solution
WriteJsonResult in case of IllegalTypeConversionException does not write any
data to the BAOS.
Such approach would need to buffer all writes to the FlowFile until full record
is processed, executing them afterwards. This may impact performance.
> WriteJsonResult leaves the ByteArrayOutputStream in inconsistent state after
> trying to write an incompatible schema record
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-14753
> URL: https://issues.apache.org/jira/browse/NIFI-14753
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.4.0
> Reporter: Dariusz Seweryn
> Priority: Minor
>
> h1. Context
> Consider a WriteJsonResult for schema ["Int Field Name": INT].
> Writing a record with ["Int Field Name": Long.MAX_VALUE] throws an
> IllegalTypeConversionException — this is expected. Unfortunately the
> ByteArrayOutputStream (BAOS) is then left with content that is not a valid
> JSON:
> { "Int Field Name" }
> See [this
> discussion|https://github.com/apache/nifi/pull/10053#discussion_r2190950596].
> h1. IllegalTypeConversionException handling
> This situation has the following consequences for exception handling apart
> from handling the offending record in the processor that tried to write the
> record:
> The same malformed record will most probably surface/cause a warning log in a
> following processor which can be surprising without the knowledge of this
> behavior in WriteJsonResult (unless all other processors assume that
> Malformed Records may appear in consumed FlowFiles and silently drop such
> records). {_}This can lead to unnecessary work being wasted by people
> triaging such warning logs{_}.
> h1. Ideal solution
> WriteJsonResult in case of IllegalTypeConversionException does not write any
> data to the BAOS.
> Such approach would need to buffer all writes to the FlowFile until full
> record is processed, executing them afterwards. This may impact performance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
