Re: PyFlink SQL: force maximum use of slots

2022-07-24 Thread Dian Fu
; *Sent:* 20 July 2022 05:19 > *To:* John Tipper > *Cc:* user@flink.apache.org > *Subject:* Re: PyFlink SQL: force maximum use of slots > > Hi John, > > All the operators in the same slot sharing group will be put in one slot. > The slot sharing group is only configurable

Re: PyFlink SQL: force maximum use of slots

2022-07-20 Thread John Tipper
copy all events to all slots, in which case I don't understand how parallelism assists? Many thanks, John From: Dian Fu Sent: 20 July 2022 05:19 To: John Tipper Cc: user@flink.apache.org Subject: Re: PyFlink SQL: force maximum use of slots Hi John

Re: PyFlink SQL: force maximum use of slots

2022-07-19 Thread Dian Fu
Hi John, All the operators in the same slot sharing group will be put in one slot. The slot sharing group is only configurable in DataStream API [1]. Usually you don't need to set the slot sharing group explicitly [2] and this is good to share the resource between the operators running in the same

PyFlink SQL: force maximum use of slots

2022-07-18 Thread John Tipper
Hi all, Is there a way of forcing a pipeline to use as many slots as possible? I have a program in PyFlink using SQL and the Table API and currently all of my pipeline is using just a single slot. I've tried this:    StreamExecutionEnvironment.get_execution_environment().disable_operator_