Re: How can I find out which key group belongs to which subtask

2020-01-13 Thread Till Rohrmann
efits. >>> As I know @Till Rohrmann has implemented some enhancements in scheduler >>> layer to support such requirement in release-1.10. You can have a try when >>> the rc candidate is ready. >>> >>> Best, >>> Zhijiang >>> >>>

Re: How can I find out which key group belongs to which subtask

2020-01-10 Thread 杨东晓
gt; Best, >> Zhijiang >> >> ------------------ >> From:杨东晓 >> Send Time:2020 Jan. 10 (Fri.) 02:10 >> To:Congxian Qiu >> Cc:user >> Subject:Re: How can I find out which key group belongs to which subtas

Re: How can I find out which key group belongs to which subtask

2020-01-10 Thread 杨东晓
> > Best, > Zhijiang > > -- > From:杨东晓 > Send Time:2020 Jan. 10 (Fri.) 02:10 > To:Congxian Qiu > Cc:user > Subject:Re: How can I find out which key group belongs to which subtask > > Thanks Congxian! > My

Re: How can I find out which key group belongs to which subtask

2020-01-10 Thread Till Rohrmann
; the rc candidate is ready. > > Best, > Zhijiang > > -- > From:杨东晓 > Send Time:2020 Jan. 10 (Fri.) 02:10 > To:Congxian Qiu > Cc:user > Subject:Re: How can I find out which key group belongs to which sub

Re: How can I find out which key group belongs to which subtask

2020-01-10 Thread Zhijiang
Subject:Re: How can I find out which key group belongs to which subtask Thanks Congxian! My purpose is not only make data goes into one same subtask but the specific subtask which belongs to same taskmanager with upstream record. The key idea is to avoid shuffling between taskmanagers. I think

Re: How can I find out which key group belongs to which subtask

2020-01-09 Thread 杨东晓
Thanks Congxian! My purpose is not only make data goes into one same subtask but the specific subtask which belongs to same taskmanager with upstream record. The key idea is to avoid shuffling between taskmanagers. I think the KeyGroupRangeAssignment.java

Re: How can I find out which key group belongs to which subtask

2020-01-09 Thread Congxian Qiu
Hi If you just want to make sure some key goes into the same subtask, does custom key selector[1] help? For the keygroup and subtask information, you can ref to KeyGroupRangeAssignment[2] for more info, and the max parallelism logic you can ref to doc[3] [1] https://ci.apache.org/projects/flink/

How can I find out which key group belongs to which subtask

2020-01-08 Thread 杨东晓
Hi , I'm trying to do some optimize about Flink 'keyby' processfunction. Is there any possible I can find out one key belongs to which key-group and essentially find out one key-group belongs to which subtask. The motivation I want to know that is we want to force the data records from upstream st