[
https://issues.apache.org/jira/browse/FLINK-38032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shuai Xu updated FLINK-38032:
-----------------------------
Description:
Follow up the test for https://issues.apache.org/jira/browse/FLINK-37874
We now support a async key ordered lookup join that performs a lookup join in
the order of specified key.
Now we'd like to have some cross team testing to validate it. We want to cover
the following cases
Test 1: Enable async key ordered lookup join, confirm the lookup join operator
in flink UI contains contents like 'async=[UNORDERED, KEY_ORDERED: true,
180000ms, 100])'. 'KEY_ORDERED: true' means the optimization is enabled.
Test 2: Verify the correctness of result. This optimization requires
non-append-only stream. For simplity, add a agg operator ahead of the lookup
join to generate cdc changelog and the upsert key which would be refered as the
order key in async key ordered lookup join. You could use the filesystem
connector for source table and lookup table.
Here are details for Test 2:
Step1. setup the envirionment and prepare the source data and lookup data.
Step2. create sql-script in the sql-client.
Step3. use [explain
statement|https://nightlies.apache.org/flink/flink-docs-master/zh/docs/dev/table/sql/explain/]
to find the upsert key before lookup join operator.
Step4. check the result. Ensure the result under the same upsert key is ordered.
was:
Follow up the test for https://issues.apache.org/jira/browse/FLINK-37874
We now support a async key ordered lookup join that performs a lookup join in
the order of specified key.
Now we'd like to have some cross team testing to validate it. We want to cover
the following cases
Test 1: Enable async key ordered lookup join, confirm the lookup join operator
in flink UI contains contents like 'async=[UNORDERED, KEY_ORDERED: true,
180000ms, 100])'. 'KEY_ORDERED: true' means the optimization is enabled.
Test 2: Verify the correctness of result. This optimization requires
non-append-only stream. For simplity, add a agg operator ahead of the lookup
join to generate cdc changelog and the upsert key which would be refered as the
order key in async key ordered lookup join. You could use the filesystem
connector for source table and lookup table.
Here are details for Test 2:
Step1. setup the envirionment.
Step2. create sql-script in the sql-client.
Step3. use [explain
statement|https://nightlies.apache.org/flink/flink-docs-master/zh/docs/dev/table/sql/explain/]
to find the upsert key before lookup join operator.
> Release Testing: Verify FLIP-519: Introduce async lookup key ordered mode
> -------------------------------------------------------------------------
>
> Key: FLINK-38032
> URL: https://issues.apache.org/jira/browse/FLINK-38032
> Project: Flink
> Issue Type: Sub-task
> Reporter: Shuai Xu
> Priority: Major
>
> Follow up the test for https://issues.apache.org/jira/browse/FLINK-37874
> We now support a async key ordered lookup join that performs a lookup join in
> the order of specified key.
> Now we'd like to have some cross team testing to validate it. We want to
> cover the following cases
> Test 1: Enable async key ordered lookup join, confirm the lookup join
> operator in flink UI contains contents like 'async=[UNORDERED, KEY_ORDERED:
> true, 180000ms, 100])'. 'KEY_ORDERED: true' means the optimization is enabled.
> Test 2: Verify the correctness of result. This optimization requires
> non-append-only stream. For simplity, add a agg operator ahead of the lookup
> join to generate cdc changelog and the upsert key which would be refered as
> the order key in async key ordered lookup join. You could use the filesystem
> connector for source table and lookup table.
> Here are details for Test 2:
> Step1. setup the envirionment and prepare the source data and lookup data.
> Step2. create sql-script in the sql-client.
> Step3. use [explain
> statement|https://nightlies.apache.org/flink/flink-docs-master/zh/docs/dev/table/sql/explain/]
> to find the upsert key before lookup join operator.
> Step4. check the result. Ensure the result under the same upsert key is
> ordered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
