[
https://issues.apache.org/jira/browse/FLINK-40346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18106685#comment-18106685
]
sepuri sai krishna commented on FLINK-40346:
--------------------------------------------
The fix for this was merged to master on 14 Aug as 969ea74 (PR #28947), but the
issue is still Open with no Fix Version set.
[~dianfu] could you resolve it and set Fix Version to 2.4.0?
One thing worth deciding at the same time: release-2.3 still carries the bug.
{{TritonTypeMapper}} on that branch has no {{containsNull}} /
{{deserializeNullableArrayFromJson}}, so {{deserializeArrayFromJson}} still
takes the primitive-backed path unconditionally and a JSON null inside an array
is read back as 0, false, or the literal string "null". {{flink-model-triton}}
ships in release-2.3, so 2.3.0 is affected as reported.
I'm happy to open a backport PR against release-2.3 if you would like this in
2.3.1.
> Triton array deserialization silently replaces JSON nulls with 0 / false /
> "null"
> ---------------------------------------------------------------------------------
>
> Key: FLINK-40346
> URL: https://issues.apache.org/jira/browse/FLINK-40346
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 2.3.0
> Reporter: sepuri sai krishna
> Assignee: sepuri sai krishna
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> {{TritonTypeMapper#deserializeArrayFromJson}} does not detect JSON {{null}}
> elements when converting a Triton inference response into {{ArrayData}}. A
> null is silently replaced by a value the model never produced:
> || Declared output type || Triton response || Deserialized result ||
> | {{ARRAY<STRING>}} | {{["a", null, "b"]}} | {{["a", "null", "b"]}} - element
> 1 is the literal 4-character string |
> | {{ARRAY<INT>}} | {{[1, null, 3]}} | {{[1, 0, 3]}} |
> | {{ARRAY<DOUBLE>}} | {{[1.5, null]}} | {{[1.5, 0.0]}} |
> | {{ARRAY<BOOLEAN>}} | {{[true, null]}} | {{[true, false]}} |
> In every case {{ArrayData#isNullAt}} returns {{false}}, so downstream
> operators cannot tell the substituted value apart from a genuine prediction.
> No exception is thrown and nothing is logged.
> The same substitution applies to the {{default-value}} connector option,
> which is parsed through the same method via
> {{TritonInferenceModelFunction#parseDefaultPayload}}: a configured fallback
> of {{[1, null, 3]}} for an {{ARRAY<INT>}} output silently becomes {{[1, 0,
> 3]}}.
> Note the asymmetry within the same class - the serialization side already
> handles this correctly ({{serializeArrayToJsonArray}} emits {{addNull()}} for
> null elements), and scalar deserialization handles it too
> ({{deserializeFromJson}} returns {{null}} for a null node). Only
> array-element deserialization is missing the check, so a null does not
> survive a serialize/deserialize round trip.
> h3. Impact
> Silent data corruption on the inference output path
> ({{TritonInferenceModelFunction#parseInferenceResponse}}). For
> {{ARRAY<STRING>}} the corrupted element is indistinguishable from a
> legitimate {{"null"}} string. For numeric arrays a missing prediction becomes
> a plausible {{0}} that flows into downstream aggregations unnoticed.
> h3. Affects
> 2.3.0 and master. The module was introduced by FLINK-38857 (fix version
> 2.3.0); the file is absent from release-2.2 and earlier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
