[
https://issues.apache.org/jira/browse/FLINK-38052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007649#comment-18007649
]
Kartikey Pant edited comment on FLINK-38052 at 7/16/25 7:31 PM:
----------------------------------------------------------------
Hi [~lsy],
I have completed the release testing for this ticket. The feature works as
expected in its primary streaming mode. Detailed test results are below.
* *Tests 1 & 2 (Correctness):* The correctness of 3-way {{INNER}} and {{LEFT}}
joins was verified using a deterministic _{{filesystem}}_ source. The results
were identical when the feature was enabled and disabled, confirming the
operator's logic.
* *Test 3 (EXPLAIN Plan):* The {{EXPLAIN}} plan was inspected for large joins
from 5-way up to 30-way. The tests were performed on {{datagen}} sources with
and without primary key constraints. In all cases, the physical plan correctly
utilized the {{MultiJoin}} operator, as expected.
* *Test 4 (Stability):* Stability was tested by executing large joins up to
30-way on _{{datagen}}_ streaming sources. The jobs all started successfully
and ran without any logic errors.
Additionally, I observed that the operator throws a {{NullPointerException}}
when the query source is a batch-style {{WITH...VALUES}} clause. I wanted to
document this observation in case it's an unintended behavior.
was (Author: JIRAUSER301503):
Hi [~lsy],
I have completed the release testing for this ticket. The feature works as
expected in its primary streaming mode. Detailed test results are below.
* *Tests 1 & 2 (Correctness): Passed.*
** *Method:* A deterministic 3-way join was performed using the
*{{filesystem}}* connector for both {{INNER}} and {{LEFT}} joins.
** *Outcome:* The materialized results were verified to be identical when
running with {{table.optimizer.multi-join.enabled}} set to {{true}} versus
{{{}false{}}}.
* *Test 3 (EXPLAIN Plan): Passed.*
** *Method:* The {{EXPLAIN}} command was used on large {{INNER JOIN}} queries
(from 5-way up to 30-way). This was tested on *{{datagen}}* sources both with
and without a declared primary key.
** *Outcome:* The resulting physical plan correctly contained the
{{MultiJoin}} operator in all cases, confirming the new optimization was being
triggered as expected.
* *Test 4 (Stability): Passed.*
** *Method:* Large-scale joins (up to 30-way) were executed live against
*{{datagen}}* streaming sources.
** *Outcome:* The jobs ran successfully without any logic errors or crashes,
demonstrating the stability of the operator.
As a side note, it was observed that the operator throws a
{{NullPointerException}} when the query source is a batch-style
{{WITH...VALUES}} clause. This may be an expected behavior or an untested edge
case, given the feature's focus on streaming mode, but I wanted to document it.
> Release Testing: Verify FLIP-516: New Streaming Multi-Way Join Operator
> -----------------------------------------------------------------------
>
> Key: FLINK-38052
> URL: https://issues.apache.org/jira/browse/FLINK-38052
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
> Affects Versions: 2.1.0
> Reporter: dalongliu
> Assignee: Kartikey Pant
> Priority: Blocker
> Labels: release-testing
> Fix For: 2.1.0
>
>
> This issue aims to verify FLIP-516.
> We now support a StreamingMultiJoinOperator that performs a join across N
> tables without intermediate state. We've done some testing and observed
> considerable performance and state improvements for multiple chained joins
> when they record amplification.
>
> Now we'd like to have some cross team testing to validate it. We want to
> cover the following cases
> * Test 1: Enable multi joins, confirm 2/3 way inner joins are supported and
> produce correct results. Materialized results should be the same as binary
> joins.
> * Test 2: Enable multi joins, confirm 2/3 way left joins are supported and
> produce correct results. Materialized results should be the same as binary
> joins.
> * Test 3: Enable 5, 10, 20, x-way joins with/without primary keys and
> confirm output plans look good with `explain`.
> * Test 4: Enable 5, 10, 20, x-way joins with/without primary keys and
> confirm they run as expected - checking results' correctness here is usually
> complicated and not strictly required.
>
> Multi joins can be enabled with ??SET 'table.optimizer.multi-join.enabled' =
> 'true';.?? Currently, we have streaming mode support for inner and left joins.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
