[
https://issues.apache.org/jira/browse/FLINK-37068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914575#comment-17914575
]
xuyang commented on FLINK-37068:
--------------------------------
Currently, FLIP-473 introduces the implementation of six commonly used
asynchronous state operators in 2.0: join, one-phase group aggregation, fast
top1, append-only top1, rowtime deduplication, window aggregation, and slicing
& one-phase window aggregation (tumble, hop, cumulative).
For testers, you can validate the correctness and functionality with large
datasets using the following steps:
Preparation Steps:
1. Environment Setup: Start a standalone cluster and launch the SQL client.
2. Set Configurations: In the SQL client, enable the asynchronous state
operator using: set 'table.exec.async-state.enabled' = 'true'. As only
one-stage group aggregation and window aggregation are currently supported,
please set: set 'table.optimizer.agg-phase-strategy' = 'ONE_PHASE' when testing
these two operators.
3. Create Tables: Execute the SQL statements in the SQL client to create the
source and result tables you want to test. You may use the filesystem connector
and CSV format for both the source and result tables to validate the
correctness of results later.
4. Prepare Data: In the SQL client, use the datagen connector to write data to
the filesystem source table.
5. Execute DML Statements: Run the DML statements that include the operators
you want to test.
Validation Steps:
1. Verify Functionality: Check the logs related to the operators in the Flink
UI. You should expect to see log entries that include the following pattern:
'xxx is using async state'.
2. Verify Result Correctness: For the same set of source table data, write to
different filesystem connector result tables by using set
'table.exec.async-state.enabled' = 'true' and set
'table.exec.async-state.enabled' = 'false', and then compare the two result
tables to see if there are any differences.
> Cross-team verification for FLIP-473
> ------------------------------------
>
> Key: FLINK-37068
> URL: https://issues.apache.org/jira/browse/FLINK-37068
> Project: Flink
> Issue Type: Sub-task
> Reporter: Xintong Song
> Assignee: xuyang
> Priority: Blocker
> Fix For: 2.0.0
>
>
> This is for cross-team verification of the release 2.0 work item: FLIP-473
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-473+Introduce+New+SQL+Operators+Based+on+Asynchronous+State+APIs
> Please complete the following steps before Jan 31.
> # Replace the description of the ticket with instructions for the testers.
> ** If the work item does not need cross-team verification, please comment
> that explicitly and close the ticket.
> # Un-assign yourself after adding the instructions.
> # Either find a cross-team tester who is willing to help and assign to the
> ticket, or reach out to the release manager [~xtsong].
> Thanks for your contributions. For any questions, feel free to reach out to
> the release manager [~xtsong].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
