[
https://issues.apache.org/jira/browse/FLINK-27541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aiden updated FLINK-27541:
--------------------------
Description:
Now we cann't add a shuffle-operation in a sql-job.
Sometimes , for example, I have a kafka-source(three partitions) with
parallelism three. And then I have a lookup-join function, I want process the
data distribute by id so that the data can split into thre parallelism evenly
(The source maybe slant seriously).
In DataStream API i can do it with keyby(), but it's so sad that i can do
nothing when i use a sql;
Maybe we can do it like 'select id, f1,f2 from sourceTable distribute by id'
like we do it in SparkSql.
was:
Now we cann't add a shuffle-operation in a sql-job.
Sometimes , for example, I have a kafka-source(three partitions) with
parallelism three. And then I have a lookup-join function, I want process the
data distribute by id so that the data can split into thre parallelism evenly
(The source maybe slant seriously). In DataStream API i can do it with
keyby(), but it's so sad that i can do nothing when i use a sql;
> Support distribute by For FlinkSql
> ----------------------------------
>
> Key: FLINK-27541
> URL: https://issues.apache.org/jira/browse/FLINK-27541
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Aiden
> Priority: Minor
>
> Now we cann't add a shuffle-operation in a sql-job.
> Sometimes , for example, I have a kafka-source(three partitions) with
> parallelism three. And then I have a lookup-join function, I want process the
> data distribute by id so that the data can split into thre parallelism evenly
> (The source maybe slant seriously).
> In DataStream API i can do it with keyby(), but it's so sad that i can do
> nothing when i use a sql;
> Maybe we can do it like 'select id, f1,f2 from sourceTable distribute by id'
> like we do it in SparkSql.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
