[
https://issues.apache.org/jira/browse/FLINK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873150#comment-17873150
]
xuyang commented on FLINK-36004:
--------------------------------
Hi, [~xccui] . This issue is triggered when Paimon outputs data to Flink,
specifically due to the custom `{{{}ArrayData`{}}} type. The method
`{{{}copy`{}}} in Flink’s `{{{}ArrayDataSerializer`{}}} does not return the
expected copied data for the custom `{{{}ArrayData`{}}}, but instead reuses its
internal data, which causes the data in the second column to overwrite that in
the first column.
Can you assign this issue to me? I plan to submit a PR soon to resolve it.
> Flink SQL returns wrong results for Paimon tables with complex schemas
> ----------------------------------------------------------------------
>
> Key: FLINK-36004
> URL: https://issues.apache.org/jira/browse/FLINK-36004
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 1.18.1, 1.19.1
> Reporter: Xingcan Cui
> Priority: Blocker
> Attachments: screenshot-1.png
>
>
> We have a Paimon table with some nested files such as the following one.
> {code:java}
> `f1` ROW <
> `f2` ROW <
> `f3` ARRAY < ROW < `key` INT, `value` FLOAT > NOT NULL >,
> `f4` ARRAY < ROW < `key` INT, `value` STRING > NOT NULL >,
> `f5` ARRAY < ROW < `key` BIGINT, `value` FLOAT > NOT NULL >,
> `f6` ARRAY < ROW < `key` BIGINT, `value` BIGINT > NOT NULL >,
> `f7` ARRAY < ROW < `key` BIGINT, `value` ROW < `f71` ARRAY <
> FLOAT > > > NOT NULL >,
> `f8` ARRAY < ROW < `f81` STRING, `f82` STRING, `f83` BIGINT >
> NOT NULL >,
> `f9` ARRAY < ROW < `key` BIGINT, `value` ROW < `f91` ARRAY <
> BIGINT > > > NOT NULL >,
> `f10` ROW < `f101` ARRAY < ROW < `f102` BIGINT, `f103`
> STRING, `f104` BIGINT > NOT NULL > >,
> `f11` ARRAY < ROW < `key` BIGINT, `value` STRING > NOT NULL >
> >
> {code}
> When a select query includes some nested columns, the results will be wrong.
> For example, {{SELECT CARDINALITY(f1.f2.f3) AS r FROM...WHERE...}} can return
> correct results but {{SELECT CARDINALITY(f1.f2.f3) AS r, f1 FROM...WHERE...}}
> will return wrong values for {{r.}}
> The query execution won't throw any exception but fails silently.
> I'm not sure if this is a Paimon-specific issue, but I also tested running
> the same query with Spark and StarRocks, and both of them can produce correct
> results.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
