[jira] [Created] (FLINK-30539) YARNSessionCapacitySchedulerITCase.testDetachedPerJobYarnCluster and testDetachedPerJobYarnClusterWithStreamingJob timing out

Matthias Pohl created FLINK-30539: - Summary: YARNSessionCapacitySchedulerITCase.testDetachedPerJobYarnCluster and testDetachedPerJobYarnClusterWithStreamingJob timing out Key: FLINK-30539 URL: https://issues.apac

[jira] [Created] (FLINK-30540) DataSinkTaskTest failed with a 143 exit code

Matthias Pohl created FLINK-30540: - Summary: DataSinkTaskTest failed with a 143 exit code Key: FLINK-30540 URL: https://issues.apache.org/jira/browse/FLINK-30540 Project: Flink Issue Type: Bu

[jira] [Created] (FLINK-30541) Add Transformer and Estimator for OnlineStandardScaler

Zhipeng Zhang created FLINK-30541: - Summary: Add Transformer and Estimator for OnlineStandardScaler Key: FLINK-30541 URL: https://issues.apache.org/jira/browse/FLINK-30541 Project: Flink Issu

Re: Flink Forward Session Question

Hi Rion, Unlike the previous Flink Forwards to the best of my knowledge the latest edition was not uploaded to YouTube. It might make sense to reach out to the authors directly. On Sat, Dec 31, 2022 at 5:35 PM Rion Williams wrote: > Hey Flinkers, > > Firstly, early Happy New Year’s to everyone

[jira] [Created] (FLINK-30542) Support adaptive hash aggregate in runtime

Yunhong Zheng created FLINK-30542: - Summary: Support adaptive hash aggregate in runtime Key: FLINK-30542 URL: https://issues.apache.org/jira/browse/FLINK-30542 Project: Flink Issue Type: Sub-

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

Hi, Jark and Dong, Thanks for your comments. Sorry for my late reply. For suggestion 1, I plan to implement the SplitEnumeratorMetricGroup in another issue, and it is not contained in this FLIP. I will add some description about this part. For suggestion 2, changes about OperatorCoordinator#metri

Re: [DISCUSS] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

Hi Junrui, Thanks for driving this FLIP, + 1 for this proposal. I believe it will greatly improve the experiences of batch users. Best, Lijie Zhu Zhu 于2022年12月30日周五 12:40写道： > Hi Junrui, > > Thanks for creating this FLIP! > > AdaptiveBatchScheduler is more powerful than DefaultScheduler in bat

Re: [DISCUSS] FLIP-266: Simplify network memory configurations for TaskManager

Hi all, Thanks for all the feedback so far. The discussion has been going on for some time. If there are no more new comments, we will start a vote today. Best, Yuxin Yuxin Tan 于2022年12月29日周四 17:37写道： > Hi, everyone > > Thanks for the reply and the discussion. > > We discussed this with @Guo

Re: [DISCUSS] Extending the feature freezing date of Flink 1.17

I'm also in favor of extending the feature freeze to Jan 31st. David On Thu, Dec 29, 2022 at 9:01 AM Leonard Xu wrote: > Thanks Qingsheng for the proposal, the pandemic has really impacted > development schedules. > > Jan 31st makes sense to me. > > > Best, > Leonard > >

Re: [EXTERNAL] Re: [DISCUSS] Add Flink Web3 Connector

Hi Martin, Thanks for the replies. But actually, Web3 represents all kinds of blockchain node. Like https://github.com/ethereum/go-ethereum, which will release Ethereum 2.0 this year. [cid:e1dcf0ae-2dde-49f2-9dbb-8d23f428a27a] Will cover sharding & indexing like ElasticSearch & Kafka. Bitcoin,

[DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

Hi devs, I'd like to start a discussion about FLINK-30234: SourceReaderBase should provide an option to disable numRecordsIn metric registration [1]. As the FLINK-302345 describes, the numRecordsIn metric is pre-registered for all sources in SourceReaderBase currently. Considering different

Re: [DISCUSS] FLIP-282: Introduce Delete & Update API

Thanks yuxia for the FLIP! It looks really good! I have three comments: ## RowLevelDeleteMode Can RowLevelDeleteMode be a higher level? `SupportsRowLevelDelete.RowLevelDeleteMode` is better than `SupportsRowLevelDelete.RowLevelDeleteInfo.RowLevelDeleteMode`. Same as `RowLevelUpdateMode`. ## Sco

[jira] [Created] (FLINK-30543) Adding more examples for setting up jobs via operator.

Sriram Ganesh created FLINK-30543: - Summary: Adding more examples for setting up jobs via operator. Key: FLINK-30543 URL: https://issues.apache.org/jira/browse/FLINK-30543 Project: Flink Issu

Re: [DISCUSS] FLIP-280: Introduce a new explain mode to provide SQL advice

Hi, Jane. Thanks for bringing this to the discussion. I have some questions about the FLIP: 1. `PlanAnalyzer#analyze` uses the FlinkRelNode as the input. Could you share some thoughts about the motivation? In my experience, users mainly care about 2 things when they develop their job: a. Why the

[jira] [Created] (FLINK-30544) Speed up finding minimum watermark across all channels by introducing heap-based algorithm

Lijie Wang created FLINK-30544: -- Summary: Speed up finding minimum watermark across all channels by introducing heap-based algorithm Key: FLINK-30544 URL: https://issues.apache.org/jira/browse/FLINK-30544

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Yunhong, Thanks for driving this~ I haven't gone deep into the implementation details yet. Regarding the general description, I would ask a few questions firstly: #1, Is there any benchmark results about the optimization latency change compared to current approach? In OLAP scenario, query opt

Re: [DISCUSS] FLIP-280: Introduce a new explain mode to provide SQL advice

Sorry for the missing answer about the configuration of the Analyzer. Users may don't need to configure this with SQL statements. In the SQL Gateway, users can configure the endpoints with the option `sql-gateway.endpoint.type` in the flink-conf. Best, Shengkai Shengkai Fang 于2023年1月3日周二 12:26写道

Re: [DISCUSS] Extending the feature freezing date of Flink 1.17

+1 to extend the feature freeze date to Jan 31st. Thanks, Zhu David Anderson 于2023年1月3日周二 11:41写道： > > I'm also in favor of extending the feature freeze to Jan 31st. > > David > > On Thu, Dec 29, 2022 at 9:01 AM Leonard Xu wrote: > > > Thanks Qingsheng for the proposal, the pandemic has really

[jira] [Created] (FLINK-30545) The SchemaManager doesn't check 'NOT NULL' specification when committing AddColumn change

yuzelin created FLINK-30545: --- Summary: The SchemaManager doesn't check 'NOT NULL' specification when committing AddColumn change Key: FLINK-30545 URL: https://issues.apache.org/jira/browse/FLINK-30545 Proje

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

Hi Benchao, Thanks for your reply. Actually, I mistakenly wrote the name "bushy join reorder" to "busy join reorder". I'm sorry for the trouble bring to you. "Bushy join reorder" means we can build a bushy join tree based on cost model, but now Flink can only build a left-deep tree using Calcite

Re: [DISCUSS] Extending the feature freezing date of Flink 1.17

+1 for the proposal (extending the 1.17 feature freeze date to Jan 31st). Best Regards, Yu On Tue, 3 Jan 2023 at 15:11, Zhu Zhu wrote: > +1 to extend the feature freeze date to Jan 31st. > > Thanks, > Zhu > > David Anderson 于2023年1月3日周二 11:41写道： > > > > I'm also in favor of extending the feat

Re: [DISCUSS] FLIP-280: Introduce a new explain mode to provide SQL advice

Thanks for driving this discussion. Do we really need to expose `PlanAnalyzerFactory` as public interface? I prefer we only expose ExplainDetail#ANALYZED_PHYSICAL_PLAN and the analyzed result. Which is enough for users and consistent with the results of `explain` method. The classes about plan an

Re: [DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

Thanks for starting the discussion, Wencong! +1 for the proposal. We can't promise that the definition of "record" in all source implementations totally matches with SourceReaderBase. For example CDC connector source [1] extracts records in the RecordEmitter instead, so the predefined "numRecordsI