Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-27 Thread Jing Ge
e scheduler, the problems you mentioned do > > exist, but I don't think it relevant here. The planner may encode some > > hints to help the scheduler, but afterall, those efforts should be > > initiated on the scheduler side. > > > > Best, > > Zhanghao Chen > > _

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-25 Thread Jing Ge
xternalized+Declarative+Resource+Management > [3] https://issues.apache.org/jira/browse/FLINK-31471 > > > Best, > Lincoln Lee > > > Chen Zhanghao 于2023年9月22日周五 16:00写道: > > > Thanks to everyone who participated in the discussion here. If no further > >

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-25 Thread Lincoln Lee
er side. > > Best, > Zhanghao Chen > ____________________ > 发件人: Lincoln Lee > 发送时间: 2023年9月22日 23:19 > 收件人: dev@flink.apache.org > 主题: Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL > Sources > > Hi Zhanghao, > > Thanks for t

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-22 Thread Lincoln Lee
message but a primary > key > > does not exist, an exception will be thrown. > > > > Best, > > Zhanghao Chen > > > > 发件人: Jane Chan > > 发送时间: 2023年9月20日 15:13 > > 收件人: dev@flink.apache.org > > 主题: Re: [DISCUSS] FLI

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-22 Thread Jane Chan
nerates update/delete data. > > > - Update the non-goals section to remove the possibly misleading > > statement that setting parallelism for individual operators lacks public > > interest and state that we leave it for future work due to its extra > > complexity.

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-20 Thread Jane Chan
your suggestions. > > > > Best, > > Zhanghao Chen > > ________________ > > 发件人: Feng Jin > > 发送时间: 2023年9月17日 0:56 > > 收件人: dev@flink.apache.org > > 主题: Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL > Sour

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-19 Thread Benchao Li
forward to your suggestions. > > Best, > Zhanghao Chen > > 发件人: Feng Jin > 发送时间: 2023年9月17日 0:56 > 收件人: dev@flink.apache.org > 主题: Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources > > Hi, Zhanghao > > Thank you

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-16 Thread Feng Jin
o use SQL hint of dynamic table options[1] to > > configure the source parallelism. > > > > [1] > > > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/#dynamic-table-options > > > > > > Best > > Yun Tang >

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-16 Thread Jane Chan
also more than glad to hear your inputs on > it. > > > > > > For Q2, scan.parallelism will take high precedence, as the more > specific > > > config should take higher precedence. > > > > > > Best, > > > Zhanghao Chen > > > __

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-15 Thread Yun Tang
e/sql/queries/hints/#dynamic-table-options Best Yun Tang From: ConradJam Sent: Friday, September 15, 2023 22:52 To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources + 1 Thanks for the FLIP and the discussi

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-15 Thread ConradJam
I think setting parallelism for each operator should also be > > > valuable. And this shall work with compiled plan [1] instead of SQL's > > DDL. > > > > > > > > > [1] > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-292%

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-15 Thread Martijn Visser
> 发件人: Jane Chan > 发送时间: 2023年9月15日 11:56 > 收件人: dev@flink.apache.org > 抄送: dewe...@outlook.com > 主题: Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL > Sources > > Hi, Zhanghao, Dewei, > > Thanks for initiatin

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-14 Thread Jane Chan
tor-level+state+TTL+configuration > > Best > Yun Tang > > From: Benchao Li > Sent: Thursday, September 14, 2023 19:53 > To: dev@flink.apache.org > Cc: dewe...@outlook.com > Subject: Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL > So

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-14 Thread Yun Tang
g Cc: dewe...@outlook.com Subject: Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources Thanks Zhanghao, Dewei for preparing the FLIP, I think this is a long awaited feature, and I appreciate your effort, especially the "Other concerns" part you listed. Regarding the

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-14 Thread Benchao Li
Thanks Zhanghao, Dewei for preparing the FLIP, I think this is a long awaited feature, and I appreciate your effort, especially the "Other concerns" part you listed. Regarding the parallelism of transformations following the source transformation, it's indeed a problem that we initially want to s