Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-22 Thread Jane Chan
the hint key cannot be omitted. E.g., the join > operator > > > is a > > > > > > TwoInputStreamOperator; if we want to specify different state > TTLs > > > for > > > > > the > > > > > > left and right inpu

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-22 Thread Benchao Li
signed to apply to the > > current > > > > > query block [2] to prevent the hint on the outer query block from > > > > > propagating to the inner query block. For the use case where users > > need > > > > to > > > > > specify the TTL for all

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-22 Thread Jane Chan
; > I share the same opinion with @Yun that this is a nice-to-have > feature. > > > Big > > > > +1 for the follow-up on FLINK-33230. Users can now use the COMPILE > PLAN > > > > statement to serialize the query to a JSON string or file and then > check

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-13 Thread Zakelly Lan
org/confluence/display/FLINK/FLIP-229%3A+Introduces+Join+Hint+for+Flink+SQL+Batch+Job > > > < > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-229%3A+Introduces+Join+Hint+for+Flink+SQL+Batch+Job > > > > > > [2] > > > > > https://nightlies.

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-13 Thread Jane Chan
che.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/#query-hints > > < > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/#query-hints > > > > > > Best, > > Jane > > > > > > On Wed, Oct 11, 2023 at 8:49 PM Yun Tang wrote: > >

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-12 Thread Zakelly Lan
e TTL for operators is a nice-to-have feature, not a > > must one in this FLIP. We can still get the information from the operator > > descriptions. > > > > And I think we can continue the TTL showing work based on FLINK-33230 [1]. > > > > Last but not least, I prefe

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-11 Thread Jane Chan
t; Best > Yun Tang > ________ > From: ConradJam > Sent: Wednesday, October 11, 2023 20:30 > To: dev@flink.apache.org > Subject: Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State > TTLs using SQL Hint > > +1 TTL shows the state tt

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-11 Thread Yun Tang
: Support Configuring Different State TTLs using SQL Hint +1 TTL shows the state ttl for operators in Flink web ui can be know what operator state Zakelly Lan 于2023年10月11日周三 19:14写道: > Hi Jane, > > The fine-grained TTL management is extremely useful for performance > tuning, so +1 for

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-11 Thread ConradJam
+1 TTL shows the state ttl for operators in Flink web ui can be know what operator state Zakelly Lan 于2023年10月11日周三 19:14写道: > Hi Jane, > > The fine-grained TTL management is extremely useful for performance > tuning, so +1 for the idea. I have a minor suggestion: would it be > possible to provi

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-11 Thread Zakelly Lan
Hi Jane, The fine-grained TTL management is extremely useful for performance tuning, so +1 for the idea. I have a minor suggestion: would it be possible to provide a simple hint that allows the omission of the key? For example, something like "SELECT /+ STATE_TTL('1h')/", which would specify the T

Re: Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-10 Thread Feng Jin
Hi Jane, Thank you for providing this explanation. Another small question, since there is no exception thrown when the STATE hint is set incorrectly, should we somehow show that the TTL setting has taken effect? For instance, exhibiting the TTL option within the operator's description? Best, Fen

Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-10 Thread Jane Chan
Hi Feng, Thank you for your valuable comments. The reason for not including the scenarios above is as follows: For <1>, the automatically inferred stateful operators are not easily expressible in SQL. This issue was discussed in FLIP-292, and besides ChangelogNormalize, SinkUpsertMateralizer also

Re: [DISCUSS] FLIP-373: Support Configuring Different State TTLs using SQL Hint

2023-10-09 Thread Feng Jin
Hi Jane, Thank you for proposing this FLIP. I believe that this FLIP will greatly enhance the flexibility of setting state, and by setting different operators' TTL, it can also increase job stability, especially in regular join scenarios. The parameter design is very concise, big +1 for this, and