Ramin Gharib created FLINK-40845:
------------------------------------
Summary: VARIANT toString() and result printing still fail on an
out-of-range TIME value
Key: FLINK-40845
URL: https://issues.apache.org/jira/browse/FLINK-40845
Project: Flink
Issue Type: Sub-task
Components: API / Core, Table SQL / Runtime
Reporter: Ramin Gharib
Assignee: Moritz Manner
FLINK-40828 made \{{Variant#toString()}} and result printing never fail. A
\{{TIME}} node whose value is outside one day still makes both of them throw.
A \{{TIME}} node stores microseconds since midnight. \{{VariantBuilder}} only
writes values in \{{[0, 86_400_000_000)}}, but a variant read from bytes
written by another engine can hold any 8-byte value.
||TIME value (micros)||\{{Variant#toString()}}||Printing
(\{{VariantCastUtils#toPrintString}})||
|\{{-1}}|throws \{{DateTimeException}}|throws \{{DateTimeException}}|
|\{{Long.MAX_VALUE}}|throws \{{DateTimeException}}|throws
\{{DateTimeException}}|
|\{{Long.MIN_VALUE}}|\{{"00:00:00"}}|\{{00:00:00.0}}|
The exception is \{{DateTimeException: Invalid value for NanoOfDay (valid
values 0 - 86399999999999): -1000}}. \{{Long.MIN_VALUE}} does not throw, but it
renders a wrong value, because \{{micros * 1000}} overflows to \{{0}}.
h3. Cause
{\{TIME}} is decoded as \{{LocalTime.ofNanoOfDay(micros * 1000)}} in
\{{BinaryVariant#getTime}} and in \{{JsonVariantFormatter#appendNode}}. Nothing
checks the range, so a bad value surfaces as a \{{DateTimeException}}. The two
lenient paths only catch \{{VariantTypeException}}:
* \{{JsonVariantFormatter#append}}, which backs \{{BinaryVariant#toString()}}.
* \{{VariantCastUtils#renderValue}} with \{{printing = true}}, which backs
\{{VariantToStringCastRule}} when printing.
The printing path has a second gap. \{{VariantCastUtils#renderScalar}} throws
\{{TableRuntimeException}} from its \{{default}} branch, also when printing.
Every current \{{Variant.Type}} has a case, but a type added later fails
printing until its case is added.
h3. Proposed fix
# Range-check \{{TIME}} where it is decoded, and throw
\{{BinaryVariantUtil#malformedVariant()}} for a value outside \{{[0,
86_400_000_000)}}. Both lenient paths then render \{{<INVALID>}}. Strict
\{{toJson()}} and \{{CAST}} fail with \{{MALFORMED_VARIANT}} instead of a
\{{DateTimeException}}. This also fixes the silent \{{Long.MIN_VALUE}} overflow.
# When printing, render a type without a case as \{{<UNKNOWN>}} instead of
throwing. \{{CAST}} keeps throwing.
h3. Tests
* \{{JsonVariantFormatterTest}}: \{{toString()}} of an array holding \{{1}} and
a \{{TIME}} of \{{-1}} micros returns \{{[1,"<INVALID>"]}}, and \{{toJson()}}
throws \{{MALFORMED_VARIANT}}.
* \{{VariantCastUtilsTest}}: \{{toPrintString}} of the same array returns
\{{[1, <INVALID>]}}.
* \{{BinaryVariantTest}}: \{{getTime()}} of that node throws
\{{MALFORMED_VARIANT}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
