[
https://issues.apache.org/jira/browse/FLINK-38029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986819#comment-17986819
]
Qilong Wang edited comment on FLINK-38029 at 6/30/25 6:12 AM:
--------------------------------------------------------------
Currently, FLIP-486 has achieved the deltaJoin with eventual consistency, which
does not require snapshot isolation for the source table data. Instead, it
always looks up the latest table data from the counterpart table during each
join operation.
For testers, you can validate the correctness and functionality with large
datasets using the following steps:
Preparation Steps:
# Environment Setup:
## Use g...@github.com:Au-Miner/fluss.git and put the compiled jar package
into Flink's lib.
## Start fluss server in standalone mode.
## Start a standalone flink cluster and launch the SQL client.
## Create fluss_catalog, the specific steps can be referred to
[fluss-docs|https://alibaba.github.io/fluss-docs/docs/install-deploy/deploying-local-cluster/].
# Set Configurations:
## In the SQL client, enable the deltaJoin operator using: set
'table.optimizer.delta-join.strategy' = 'auto'. Of course, if you want to turn
it off or force it on, you can set it to 'none' or 'force'.
# Create Tables:
## You can use datagen to import the generated data into fluss table by
creating a temporary table.
## Since deltaJoin currently only supports append-only streams, you need to
set the fluss table to table.merge-engine'='first_row' or use a log table.
## You may use the filesystem connector and CSV format for result table to
validate the correctness of results later.
# Execute DML Statements:
## Execute DML Statements: Run the DML statements that include the deltajoin
operator you want to test.
## Please refer to FLINK-37897 for the patterns currently supported by
deltaJoin.
Validation Steps:
# Verify Functionality: Check the operator icon related to the operators in
the Flink UI. You should expect to see the deltaJoin operator icon.
# Verify Result Correctness: For the same set of source table data, write to
two result tables by using set 'table.optimizer.delta-join.strategy' = 'auto'
and set 'table.optimizer.delta-join.strategy' = 'none', and then compare the
two result tables to see if there are any differences.
was (Author: JIRAUSER306284):
Currently, FLIP-486 has achieved the deltaJoin with eventual consistency, which
does not require snapshot isolation for the source table data. Instead, it
always looks up the latest table data from the counterpart table during each
join operation.
For testers, you can validate the correctness and functionality with large
datasets using the following steps:
Preparation Steps:
# Environment Setup:
## Use g...@github.com:Au-Miner/fluss.git and put the compiled jar package
into Flink's lib.
## Start fluss server in standalone mode.
## Start a standalone flink cluster and launch the SQL client.
## Create fluss_catalog, the specific steps can be referred to
[fluss-docs|https://alibaba.github.io/fluss-docs/docs/install-deploy/deploying-local-cluster/].
# Set Configurations:
## In the SQL client, enable the deltaJoin operator using: set
'table.optimizer.delta-join.strategy' = 'auto'. Of course, if you want to turn
it off or force it on, you can set it to 'none' or 'force'.
# Create Tables:
## You can use datagen to import the generated data into fluss table by
creating a temporary table.
## Since deltaJoin currently only supports append-only streams, you need to
set the fluss table to table.merge-engine'='first_row' or use a log table.
## You may use the filesystem connector and CSV format for result table to
validate the correctness of results later.
# Execute DML Statements:
## Execute DML Statements: Run the DML statements that include the deltajoin
operator you want to test.
## Please refer to FLINK-37897 for the patterns currently supported by
deltaJoin.
Validation Steps:
# Verify Functionality: Check the operator icon related to the operators in
the Flink UI. You should expect to see the deltaJoin operator icon.
# Verify Result Correctness: For the same set of source table data, write to
two result tables by using set 'table.optimizer.delta-join.strategy' = 'auto'
and set 'table.optimizer.delta-join.strategy' = 'none', and then compare the
two result tables to see if there are any differences.
> Release Testing: Verify FLIP-486: Introduce a new DeltaJoin
> -----------------------------------------------------------
>
> Key: FLINK-38029
> URL: https://issues.apache.org/jira/browse/FLINK-38029
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner, Table SQL / Runtime
> Affects Versions: 2.1.0
> Reporter: Xuyang Zhong
> Priority: Blocker
>
> Follow up the test for https://issues.apache.org/jira/browse/FLINK-37836
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
