Hi all,
Currently, when performing cascading connections in Flink, there is a pain
point of record amplification as mentioned in discussion similar to
https://lists.apache.org/thread/2021fmwhtotl0okmtyc5b7tndlp3khf9,
I have implemented the optimization of POC and it works. I wonder to forward a
Hi devs,
I’d like to start a discussion on FLIP-415: Introduce a new join operator to
support minibatch[1].
Currently, when performing cascading connections in Flink, there is a pain
point of record amplification. Every record join operator receives would
trigger join process. However, if reco
or users, harder to understand and
> maintain, which we should be careful about.
>
> One thing about the implementation, could you make the new operator
> share the same state definition with the original one?
>
> shuai xu 于2024年1月10日周三 21:23写道：
>>
>> Hi devs,
>
original
>> Join and mini-batch Join from this aspect. Besides, introducing more
>> options would make it more complex for users, harder to understand and
>> maintain, which we should be careful about.
>>
>> One thing about the implementation, could you make the
Hi Leonard,
Thanks for your help very much !
I have already started the discussion about FLIP-415: Introduce a new join
operator to support minibatch[1].
And I’m looking forward to your feedback if you have some spare time to take a
look.
[1]
https://cwiki.apache.org/confluence/display/FLINK
downstream of the job which
>> requires details of changelog
>
> Could you elaborate on this a bit? I've never met such kinds of
> requirements before, I'm curious what is the scenario that requires
> this.
>
> shuai xu 于2024年1月11日周四 13:08写道：
>>
>>
to join to control CDC message folding.
>>
>>
>>
>>
>> --
>>
>>Best！
>>Xuyang
>>
>>
>>
>>
>>
>> 在 2024-01-11 16:19:30，"Benchao Li" 写道：
>>>> the change might not be suppo
Hi all,
CEP is broadly used in more and more applications now. In may cases, users
need to use the pattern CEP.begin().notFollowedBy(). For example, they may
want to get the uses who created an oder but didn't pay in 10 minutes and
so on.
However, CEP doesn't support notFollowBy() as the last part
Hi all
Glad to see the discussion, we are now designing to enhance the scheduling
of batch job, a unified api will help a lot.
Haibo Sun 于2018年12月5日周三 下午4:45写道：
> Hi all,
>
> Thank Kurt, you see more benefits of the unification than I do.
>
> I quite agree Kurt's views. DataStream, DataSet and T
Will we only remove command support in client side or the code in job
master will also be removed?
Till Rohrmann 于2019年4月24日周三 下午4:12写道：
> +1 for temporarily removing support for the modify command.
>
> Eventually, we have to add it again in order to support auto scaling. The
> next time we add
Hi Benchao,
Do you have any other questions about this issue? Also, I would appreciate
your thoughts on the proposal to introduce the new option
'table.exec.mini-batch.compact-changes-enabled'. I’m looking forward your
feedback.
> 2024年1月12日 15:01，shuai xu 写道：
>
> Suppos
gt; Currently there is no standard about whether a change is major or not,
> which means we may run into a situation debating whether a change is
> major or not. Anyway, it's an orthogonal topic to this discussion.
>
> shuai xu 于2024年1月16日周二 13:14写道：
>>
>> Hi Benchao,
ompaction within the minibatch that was mentioned in the discussion, it
could be discussed in a future FLIP.
Do any of you have further questions regarding this FLIP? If there are no more
comments, I would like to open a voting thread at 12 a.m. UTC+8 on January
19th.
> 2024年1月10日 21:23，shuai xu 写道：
Dear Flink Developers,
Thank you for providing feedback on FLIP-415: Introduce a new join operator to
support minibatch[1]. I'd like to start a vote on this FLIP. Here is the
discussion thread[2].
After the discussion, this FLIP will not introduce any new Option. The
minibatch join will defau
Hi devs,
I'm glad to announce that the FLIP-415[1] has been accepted. The voting
thread is here[2].
The proposal received 5 approving votes, 4 of which are binding:
- Lincoln Lee (binding)
- Ron liu (binding)
- Jane Chan (binding)
- Benchao Li (binding)
- Xuyang (non-binding)
And there is no di
Hi devs,
I'm glad to announce that the FLIP-415[1] has been accepted. The voting
thread is here[2].
The proposal received 5 approving votes, all of which are as follow:
- Lincoln Lee (binding)
- liu ron (binding)
- Jane Chan (binding)
- Benchao Li (binding)
- Xuyang (non-binding)
And there is no
Hi devs,
I'm glad to announce that the FLIP-415[1] has been accepted. The voting
thread is here[2].
The proposal received five approving votes, four of which are binding:
- Lincoln Lee (binding)
- Jane Chan (binding)
- Ron liu (binding)
- Benchao Li (binding)
- Xuyang (non-binding)
And there i
Hi everyone,
I am currently working on FLIP-415[1], which aims to support minibatch
join. This will bring higher performance for regular joins.
There is still a task to be merged which has undergone multiple rounds
of review. I expect it could be merged on Jan 26th.
Therefore, I am +1 with Feng J
Hi, Roman
Thanks for your proposal. I think this is an interesting idea and it might be
useful when there are operators downstream of the TopN.
And I have some questions about your proposal after reading your doc.
1. From the input-output perspective, only the accumulated data seems to be
sent
+1（not-binding）
Best,
Xushuai
> 2024年11月13日 10:48，Zhu Zhu 写道：
>
> +1 (binding)
>
> Thanks,
> Zhu
>
> Ferenc Csaky 于2024年11月13日周三 02:37写道：
>
>> +1 (binding)
>>
>> Best,
>> Ferenc
>>
>>
>>
>>
>> On Tuesday, November 12th, 2024 at 03:28, Xuyang
>> wrote:
>>
>>>
>>>
>>> +1 (non-binding
Shuai Xu created FLINK-32219:
Summary: sql client would be pending after executing plan of
inserting
Key: FLINK-32219
URL: https://issues.apache.org/jira/browse/FLINK-32219
Project: Flink
Issue
Shuai Xu created FLINK-33689:
Summary: jsonObjectAggFunction can't retract previous data which
is invalid when enable local global agg
Key: FLINK-33689
URL: https://issues.apache.org/jira/browse/FLINK-
Shuai Xu created FLINK-34219:
Summary: Introduce a new join operator to support minibatch
Key: FLINK-34219
URL: https://issues.apache.org/jira/browse/FLINK-34219
Project: Flink
Issue Type: New
Shuai Xu created FLINK-34220:
Summary: introduce buffer bundle for minibatch join
Key: FLINK-34220
URL: https://issues.apache.org/jira/browse/FLINK-34220
Project: Flink
Issue Type: Sub-task
Shuai Xu created FLINK-34221:
Summary: Introduce operator for minibatch join
Key: FLINK-34221
URL: https://issues.apache.org/jira/browse/FLINK-34221
Project: Flink
Issue Type: Sub-task
Shuai Xu created FLINK-34222:
Summary: Get minibatch join operator involved
Key: FLINK-34222
URL: https://issues.apache.org/jira/browse/FLINK-34222
Project: Flink
Issue Type: Sub-task
Shuai Xu created FLINK-34256:
Summary: Add a documentation section for minibatch join
Key: FLINK-34256
URL: https://issues.apache.org/jira/browse/FLINK-34256
Project: Flink
Issue Type: Sub-task
Shuai Xu created FLINK-34349:
Summary: Release Testing: Verify FLINK-34219 Introduce a new join
operator to support minibatch
Key: FLINK-34349
URL: https://issues.apache.org/jira/browse/FLINK-34349
Shuai Xu created FLINK-34462:
Summary: Session window with negative parameter throws unclear
exception
Key: FLINK-34462
URL: https://issues.apache.org/jira/browse/FLINK-34462
Project: Flink
Shuai Xu created FLINK-36654:
Summary: Decimal divide Integer reports Null pointer exception
Key: FLINK-36654
URL: https://issues.apache.org/jira/browse/FLINK-36654
Project: Flink
Issue Type
Shuai Xu created FLINK-36962:
Summary: push down non-deterministic filter after stream join to
source by mistake
Key: FLINK-36962
URL: https://issues.apache.org/jira/browse/FLINK-36962
Project: Flink
31 matches
Mail list logo
