Re: [DISCUSS] FLIP-330: Support specifying record timestamp requirement

Hey Devs, It would be great to see some other benchmarks , not only the dummy WordCount example. I would love to see a few SQL queries documented and whether there is any measurable benefit at all. Prod pipelines usually have some IO component etc which will add enough overhead to make this eve

Re: [DISCUSS] FLIP-330: Support specifying record timestamp requirement

Thanks for the FLIP, Yunfeng. I had a brief offline discussion with Dong, and here are my two cents: ## The benefit The FLIP is related to one of the perf benchmarks we saw at LinkedIn which is pretty much doing a word count, except that the words are country code, so it is typically just two byt

Re: [DISCUSS] Status of Statefun Project

I'm done with the code to make Statefun compatible with Flink 1.16, and all the tests (including e2e succeed). The required changes were pretty minimal. I'm running into a bit of a chicken/egg problem executing the tests in flink-statefun-playground

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

Congratulations, Matthias Bests, Samrat On Wed, Aug 9, 2023 at 4:18 PM Benchao Li wrote: > Congratulations, Matthias! > > Maximilian Michels 于2023年8月8日周二 17:54写道： > > > Congrats, well done, and welcome to the PMC Matthias! > > > > -Max > > > > On Tue, Aug 8, 2023 at 8:36 AM yh z wrote: > > >

Re: [ANNOUNCE] New Apache Flink Committer - Hong Teoh

Congratulations Hong Bests, Samrat On Fri, Aug 4, 2023 at 2:37 PM Dan Zou wrote: > Congrats, Hong! > > Best, > Dan Zou > > > > > > > 2023年8月4日 15:30，Lijie Wang 写道： > > > > Congratulations, Hong! > > > > Best, > > Lijie > > > > Paul Lam 于2023年8月4日周五 15:18写道： > > > >> Congrats, Hong! > >> > >>

Re: [DISCUSS] FLIP-330: Support specifying record timestamp requirement

Hi Xintong, Thanks for the quick reply. I also agree that we should hear from others about whether this optimization is worthwhile. Please see my comments inline. On Fri, Aug 11, 2023 at 5:54 PM Xintong Song wrote: > Thanks for the quick replies. > > Overall, it seems that the main concern wit

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

Hi Piotr, Thanks for the reply! On Fri, Aug 11, 2023 at 4:44 PM Piotr Nowojski wrote: > Hi, > > Sorry for the long delay in responding! > > > Given that it is an optional feature that can be > > turned off by users, it might be OK to just let users try it out and we > can > > fix performance i

Re: [DISCUSS] FLIP-330: Support specifying record timestamp requirement

Thanks for the quick replies. Overall, it seems that the main concern with this FLIP is that the 2% > throughput saving might not be worth the added implementation complexity. > Yes, my main concern is that the performance improvement may not be worth the added complexity. And I'd like to point

[jira] [Created] (FLINK-32858) [JUnit5 Migration] The util package of flink-runtime module

Rui Fan created FLINK-32858: --- Summary: [JUnit5 Migration] The util package of flink-runtime module Key: FLINK-32858 URL: https://issues.apache.org/jira/browse/FLINK-32858 Project: Flink Issue Type

[jira] [Created] (FLINK-32857) [JUnit5 Migration] The webmonitor and zookeeper packages of flink-runtime module

Rui Fan created FLINK-32857: --- Summary: [JUnit5 Migration] The webmonitor and zookeeper packages of flink-runtime module Key: FLINK-32857 URL: https://issues.apache.org/jira/browse/FLINK-32857 Project: Flink

[jira] [Created] (FLINK-32856) [JUnit5 Migration] The testtasks, testutils, throughput and throwable packages of flink-runtime module

Rui Fan created FLINK-32856: --- Summary: [JUnit5 Migration] The testtasks, testutils, throughput and throwable packages of flink-runtime module Key: FLINK-32856 URL: https://issues.apache.org/jira/browse/FLINK-32856

[jira] [Created] (FLINK-32854) [JUnit5 Migration] The state package of flink-runtime module

Rui Fan created FLINK-32854: --- Summary: [JUnit5 Migration] The state package of flink-runtime module Key: FLINK-32854 URL: https://issues.apache.org/jira/browse/FLINK-32854 Project: Flink Issue Typ

[jira] [Created] (FLINK-32855) [JUnit5 Migration] The taskexecutor package of flink-runtime module

Rui Fan created FLINK-32855: --- Summary: [JUnit5 Migration] The taskexecutor package of flink-runtime module Key: FLINK-32855 URL: https://issues.apache.org/jira/browse/FLINK-32855 Project: Flink Is

[jira] [Created] (FLINK-32853) [JUnit5 Migration] The security, taskmanager and source packages of flink-runtime module

Rui Fan created FLINK-32853: --- Summary: [JUnit5 Migration] The security, taskmanager and source packages of flink-runtime module Key: FLINK-32853 URL: https://issues.apache.org/jira/browse/FLINK-32853 Projec

[jira] [Created] (FLINK-32852) [JUnit5 Migration] The scheduler package of flink-runtime module

Rui Fan created FLINK-32852: --- Summary: [JUnit5 Migration] The scheduler package of flink-runtime module Key: FLINK-32852 URL: https://issues.apache.org/jira/browse/FLINK-32852 Project: Flink Issue

[jira] [Created] (FLINK-32850) [JUnit5 Migration] The io package of flink-runtime module

Rui Fan created FLINK-32850: --- Summary: [JUnit5 Migration] The io package of flink-runtime module Key: FLINK-32850 URL: https://issues.apache.org/jira/browse/FLINK-32850 Project: Flink Issue Type: S

[jira] [Created] (FLINK-32851) [JUnit5 Migration] The rest package of flink-runtime module

Rui Fan created FLINK-32851: --- Summary: [JUnit5 Migration] The rest package of flink-runtime module Key: FLINK-32851 URL: https://issues.apache.org/jira/browse/FLINK-32851 Project: Flink Issue Type

[jira] [Created] (FLINK-32849) [JUnit5 Migration] The resourcemanager package of flink-runtime module

Rui Fan created FLINK-32849: --- Summary: [JUnit5 Migration] The resourcemanager package of flink-runtime module Key: FLINK-32849 URL: https://issues.apache.org/jira/browse/FLINK-32849 Project: Flink

[jira] [Created] (FLINK-32848) [JUnit5 Migration] The persistence, query, registration, rpc and shuffle packages of flink-runtime module

Rui Fan created FLINK-32848: --- Summary: [JUnit5 Migration] The persistence, query, registration, rpc and shuffle packages of flink-runtime module Key: FLINK-32848 URL: https://issues.apache.org/jira/browse/FLINK-32848

[jira] [Created] (FLINK-32847) [JUnit5 Migration] The operators package of flink-runtime module

Rui Fan created FLINK-32847: --- Summary: [JUnit5 Migration] The operators package of flink-runtime module Key: FLINK-32847 URL: https://issues.apache.org/jira/browse/FLINK-32847 Project: Flink Issue

[jira] [Created] (FLINK-32846) [JUnit5 Migration] The metrics, minicluster and net packages of flink-runtime module

Rui Fan created FLINK-32846: --- Summary: [JUnit5 Migration] The metrics, minicluster and net packages of flink-runtime module Key: FLINK-32846 URL: https://issues.apache.org/jira/browse/FLINK-32846 Project: F

[jira] [Created] (FLINK-32845) [JUnit5 Migration] The leaderelection, leaderretrieval, mailbox, memory and messages packages of flink-runtime module

Rui Fan created FLINK-32845: --- Summary: [JUnit5 Migration] The leaderelection, leaderretrieval, mailbox, memory and messages packages of flink-runtime module Key: FLINK-32845 URL: https://issues.apache.org/jira/browse/FL

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

Hi, Sorry for the long delay in responding! > Given that it is an optional feature that can be > turned off by users, it might be OK to just let users try it out and we can > fix performance issues once we detect any of them. What do you think? I think it's fine. It would be best to mark this f

[jira] [Created] (FLINK-32843) [JUnit5 Migration] The jobmaster package of flink-runtime module

Rui Fan created FLINK-32843: --- Summary: [JUnit5 Migration] The jobmaster package of flink-runtime module Key: FLINK-32843 URL: https://issues.apache.org/jira/browse/FLINK-32843 Project: Flink Issue

[jira] [Created] (FLINK-32844) Runtime Filter should not be applied if the field is already filtered by DPP

Lijie Wang created FLINK-32844: -- Summary: Runtime Filter should not be applied if the field is already filtered by DPP Key: FLINK-32844 URL: https://issues.apache.org/jira/browse/FLINK-32844 Project: Fli

[jira] [Created] (FLINK-32842) [JUnit5 Migration] The iterative, jobgraph and jobmanager packages of flink-runtime module

Rui Fan created FLINK-32842: --- Summary: [JUnit5 Migration] The iterative, jobgraph and jobmanager packages of flink-runtime module Key: FLINK-32842 URL: https://issues.apache.org/jira/browse/FLINK-32842 Proj

[jira] [Created] (FLINK-32841) [JUnit5 Migration] The failure, filecache, hadoop, heartbeat, highavailability and instancepackages of flink-runtime module

Rui Fan created FLINK-32841: --- Summary: [JUnit5 Migration] The failure, filecache, hadoop, heartbeat, highavailability and instancepackages of flink-runtime module Key: FLINK-32841 URL: https://issues.apache.org/jira/bro

[jira] [Created] (FLINK-32840) [JUnit5 Migration] The executiongraph package of flink-runtime module

Rui Fan created FLINK-32840: --- Summary: [JUnit5 Migration] The executiongraph package of flink-runtime module Key: FLINK-32840 URL: https://issues.apache.org/jira/browse/FLINK-32840 Project: Flink

[jira] [Created] (FLINK-32839) [JUnit5 Migration] The entrypoint, event, execution and externalresource packages of flink-runtime module

Rui Fan created FLINK-32839: --- Summary: [JUnit5 Migration] The entrypoint, event, execution and externalresource packages of flink-runtime module Key: FLINK-32839 URL: https://issues.apache.org/jira/browse/FLINK-32839

[jira] [Created] (FLINK-32838) [JUnit5 Migration] The deployment and dispatcher packages of flink-runtime module

Rui Fan created FLINK-32838: --- Summary: [JUnit5 Migration] The deployment and dispatcher packages of flink-runtime module Key: FLINK-32838 URL: https://issues.apache.org/jira/browse/FLINK-32838 Project: Flin

[jira] [Created] (FLINK-32837) [JUnit5 Migration] The client, clusterframework and concurrent packages of flink-runtime module

Rui Fan created FLINK-32837: --- Summary: [JUnit5 Migration] The client, clusterframework and concurrent packages of flink-runtime module Key: FLINK-32837 URL: https://issues.apache.org/jira/browse/FLINK-32837

[jira] [Created] (FLINK-32836) [JUnit5 Migration] The checkpoint package of flink-runtime module

Rui Fan created FLINK-32836: --- Summary: [JUnit5 Migration] The checkpoint package of flink-runtime module Key: FLINK-32836 URL: https://issues.apache.org/jira/browse/FLINK-32836 Project: Flink Issu

[jira] [Created] (FLINK-32835) [JUnit5 Migration] The accumulators, blob and blocklist package of flink-runtime module

Rui Fan created FLINK-32835: --- Summary: [JUnit5 Migration] The accumulators, blob and blocklist package of flink-runtime module Key: FLINK-32835 URL: https://issues.apache.org/jira/browse/FLINK-32835 Project

[jira] [Created] (FLINK-32834) Allow compile.sh to be used manually

Chesnay Schepler created FLINK-32834: Summary: Allow compile.sh to be used manually Key: FLINK-32834 URL: https://issues.apache.org/jira/browse/FLINK-32834 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-32833) Rocksdb CacheIndexAndFilterBlocks must be true when using shared memory

Yue Ma created FLINK-32833: -- Summary: Rocksdb CacheIndexAndFilterBlocks must be true when using shared memory Key: FLINK-32833 URL: https://issues.apache.org/jira/browse/FLINK-32833 Project: Flink