[jira] [Created] (FLINK-36974) support overwrite flink config by command line

2024-12-27 Thread hiliuxg (Jira)
hiliuxg created FLINK-36974: --- Summary: support overwrite flink config by command line Key: FLINK-36974 URL: https://issues.apache.org/jira/browse/FLINK-36974 Project: Flink Issue Type: New Feature

[PR] support overwrite flink config by command line [flink-cdc]

2024-12-27 Thread via GitHub
hiliuxg opened a new pull request, #3823: URL: https://github.com/apache/flink-cdc/pull/3823 Support overwrite flink config in the command line, for example: `bin/flink-cdc.sh1732864461789.yaml --flink-conf execution.checkpointing.interval=10min --flink-conf rest.bind-port=42689 --fl

[jira] [Updated] (FLINK-36973) udf dateformat support LocalZonedTimestampData datatype

2024-12-27 Thread hiliuxg (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hiliuxg updated FLINK-36973: External issue URL: https://github.com/apache/flink-cdc/pull/3822 Summary: udf dateformat su

[jira] [Updated] (FLINK-36973) udf dateformat support LocalZonedTimestampData

2024-12-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36973: --- Labels: pull-request-available (was: ) > udf dateformat support LocalZonedTimestampData > -

[PR] [FLINK-36973]udf dateformat support LocalZonedTimestampData [flink-cdc]

2024-12-27 Thread via GitHub
hiliuxg opened a new pull request, #3822: URL: https://github.com/apache/flink-cdc/pull/3822 udf dateformat support LocalZonedTimestampData datatype -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[jira] [Created] (FLINK-36973) udf dateformat support LocalZonedTimestampData

2024-12-27 Thread hiliuxg (Jira)
hiliuxg created FLINK-36973: --- Summary: udf dateformat support LocalZonedTimestampData Key: FLINK-36973 URL: https://issues.apache.org/jira/browse/FLINK-36973 Project: Flink Issue Type: New Feature

[jira] [Closed] (FLINK-36069) Extending job detail rest API to adapt to Incremental JobGraph Generation

2024-12-27 Thread Junrui Lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junrui Lee closed FLINK-36069. -- Fix Version/s: 2.0.0 Resolution: Fixed master 08990c7d21260dff55e3a499354f5a5980ae68e6 > Exten

Re: [PR] [FLINK-36069][runtime/rest] Extending job detail rest API to expose json stream graph [flink]

2024-12-27 Thread via GitHub
JunRuiLee merged PR #25798: URL: https://github.com/apache/flink/pull/25798 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] Remove CHARACTER_FILTER.filterCharacters function when notify dimension values. [flink]

2024-12-27 Thread via GitHub
hiliuxg commented on PR #25710: URL: https://github.com/apache/flink/pull/25710#issuecomment-2564165517 > Reviewed by Chi on 05/12/24. Asked submitter questions Hi, @davidradl I have added a unit test. Could you please review the code again? thank you -- This is an automated mess

Re: [PR] [FLINK-36069][runtime/rest] Extending job detail rest API to expose json stream graph [flink]

2024-12-27 Thread via GitHub
yuchen-ecnu commented on PR #25798: URL: https://github.com/apache/flink/pull/25798#issuecomment-2564142724 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [FLINK-36027][cdc-connector][mysql] numRecordsOut/numRecordsOutRate metrics for each OperationType [flink-cdc]

2024-12-27 Thread via GitHub
github-actions[bot] commented on PR #3529: URL: https://github.com/apache/flink-cdc/pull/3529#issuecomment-2564107452 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

[jira] [Commented] (FLINK-36951) Migrate ProjectSemiAntiJoinTransposeRule

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908518#comment-17908518 ] Sergey Nuyanzin commented on FLINK-36951: - Merged as [cb75a588398ffcb4e8f705508

[jira] [Resolved] (FLINK-36951) Migrate ProjectSemiAntiJoinTransposeRule

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin resolved FLINK-36951. - Assignee: Jacky Lau Resolution: Fixed > Migrate ProjectSemiAntiJoinTransposeRule >

Re: [PR] [FLINK-36951][table] Migrate ProjectSemiAntiJoinTransposeRule to java [flink]

2024-12-27 Thread via GitHub
snuyanzin merged PR #25844: URL: https://github.com/apache/flink/pull/25844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

[jira] [Commented] (FLINK-36954) Migrate CalcRankTransposeRule

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908517#comment-17908517 ] Sergey Nuyanzin commented on FLINK-36954: - Merged as [e26d721feaeaf253e50e3c2a8

[jira] [Resolved] (FLINK-36954) Migrate CalcRankTransposeRule

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin resolved FLINK-36954. - Assignee: Jacky Lau Resolution: Fixed > Migrate CalcRankTransposeRule > --

Re: [PR] [FLINK-36954][table] Migrate CalcRankTransposeRule to java [flink]

2024-12-27 Thread via GitHub
snuyanzin merged PR #25846: URL: https://github.com/apache/flink/pull/25846 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

[jira] [Resolved] (FLINK-36955) Migrate RewriteIntersectAllRule

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin resolved FLINK-36955. - Assignee: Jacky Lau Resolution: Fixed > Migrate RewriteIntersectAllRule >

[jira] [Commented] (FLINK-36955) Migrate RewriteIntersectAllRule

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908516#comment-17908516 ] Sergey Nuyanzin commented on FLINK-36955: - Merged as [59ef8d6af85f374cbd34a92eb

Re: [PR] [FLINK-36955][table] Migrate RewriteIntersectAllRule to java [flink]

2024-12-27 Thread via GitHub
snuyanzin merged PR #25847: URL: https://github.com/apache/flink/pull/25847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] [FLINK-35282][python] Upgrade Apache Beam > 2.54 [flink]

2024-12-27 Thread via GitHub
franciscojavierarceo commented on PR #25541: URL: https://github.com/apache/flink/pull/25541#issuecomment-2563974146 Any update on this? @dianfu @HuangXingBo @snuyanzin ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [FLINK-36146][connector] Fix SingleThreadFetcherManager race condition [flink]

2024-12-27 Thread via GitHub
kimgr commented on PR #25340: URL: https://github.com/apache/flink/pull/25340#issuecomment-2563956494 @becketqin Gentle holiday ping! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [FLINK-33117][docs] Fix scala example for `NamedParameterClass` [flink]

2024-12-27 Thread via GitHub
snuyanzin merged PR #25840: URL: https://github.com/apache/flink/pull/25840 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

[jira] [Resolved] (FLINK-34675) Migrate AggregateReduceGroupingRule

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin resolved FLINK-34675. - Fix Version/s: 2.0.0 Resolution: Fixed Merged as [ada277e90d369aa357e5b1a0c4fa91

Re: [PR] [FLINK-34675][table] Migrate AggregateReduceGroupingRule to java [flink]

2024-12-27 Thread via GitHub
snuyanzin merged PR #24502: URL: https://github.com/apache/flink/pull/24502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] [hotfix][docs] Fix typo in Amazon S3 Access Credential [flink]

2024-12-27 Thread via GitHub
snuyanzin merged PR #25857: URL: https://github.com/apache/flink/pull/25857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

[jira] [Commented] (FLINK-36966) Bump mockito to 5.x

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908503#comment-17908503 ] Sergey Nuyanzin commented on FLINK-36966: - Merged as [1e9e848fae9b989f1503bc073

[jira] [Resolved] (FLINK-36966) Bump mockito to 5.x

2024-12-27 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin resolved FLINK-36966. - Fix Version/s: 2.0.0 Resolution: Fixed > Bump mockito to 5.x > --

Re: [PR] [FLINK-36966][tests] Bump mockito to 5.14.2 [flink]

2024-12-27 Thread via GitHub
snuyanzin merged PR #25862: URL: https://github.com/apache/flink/pull/25862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

[jira] [Created] (FLINK-36972) Unify AWS sink implementation

2024-12-27 Thread Ahmed Hamdy (Jira)
Ahmed Hamdy created FLINK-36972: --- Summary: Unify AWS sink implementation Key: FLINK-36972 URL: https://issues.apache.org/jira/browse/FLINK-36972 Project: Flink Issue Type: Technical Debt

[jira] [Updated] (FLINK-36971) Add Sqs Sink SQL connector

2024-12-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36971: --- Labels: pull-request-available (was: ) > Add Sqs Sink SQL connector > -

[PR] [FLINK-36971] Add Sqs SQL connector [flink-connector-aws]

2024-12-27 Thread via GitHub
vahmed-hamdy opened a new pull request, #187: URL: https://github.com/apache/flink-connector-aws/pull/187 ## Purpose of the change *Add SQS SQL connector wrapper for table api connector* ## Verifying this change - The main functionality is covered by Table API tes

Re: [PR] [FLINK-36931][cdc] FlinkCDC YAML supports batch mode [flink-cdc]

2024-12-27 Thread via GitHub
aiwenmo commented on PR #3812: URL: https://github.com/apache/flink-cdc/pull/3812#issuecomment-2563877827 5. DataSource will send CreateTableCompletedEvent after sending all CreateTableEvent. 6. add CreateTableCompletedEvent to notify SchemaBatchOperator to merge all CreateTableEvent.

[jira] [Created] (FLINK-36971) Add Sqs Sink SQL connector

2024-12-27 Thread Ahmed Hamdy (Jira)
Ahmed Hamdy created FLINK-36971: --- Summary: Add Sqs Sink SQL connector Key: FLINK-36971 URL: https://issues.apache.org/jira/browse/FLINK-36971 Project: Flink Issue Type: Improvement Co

[jira] [Updated] (FLINK-36971) Add Sqs Sink SQL connector

2024-12-27 Thread Ahmed Hamdy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hamdy updated FLINK-36971: Description: h1. Description - After adding table api connector for sqs sink, we want to add SQL c

Re: [PR] [FLINK-36970][runtime] Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING [flink-cdc]

2024-12-27 Thread via GitHub
yuxiqian commented on code in PR #3821: URL: https://github.com/apache/flink-cdc/pull/3821#discussion_r1898394493 ## flink-cdc-common/src/test/java/org/apache/flink/cdc/common/utils/SchemaMergingUtilsTest.java: ## Review Comment: There are some outdated comments in L978 and

Re: [PR] [FLINK-36961][state/forst] Wait for the ForSt state executor to shutdown when disposing [flink]

2024-12-27 Thread via GitHub
fredia commented on PR #25854: URL: https://github.com/apache/flink/pull/25854#issuecomment-2563564398 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] [FLINK-36825][table] Introduce FastTop1Function in Rank with Async State API [flink]

2024-12-27 Thread via GitHub
xuyangzhong commented on code in PR #25717: URL: https://github.com/apache/flink/pull/25717#discussion_r1898410221 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/rank/FastTop1Function.java: ## @@ -30,38 +30,28 @@ import org.apache.flin

Re: [PR] [FLINK-36576][runtime] Improving amount-based data balancing distribution algorithm for DefaultVertexParallelismAndInputInfosDecider [flink]

2024-12-27 Thread via GitHub
zhuzhurk commented on code in PR #25552: URL: https://github.com/apache/flink/pull/25552#discussion_r1898346551 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptivebatch/BlockingInputInfo.java: ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] [FLINK-36825][table] Introduce FastTop1Function in Rank with Async State API [flink]

2024-12-27 Thread via GitHub
xuyangzhong commented on code in PR #25717: URL: https://github.com/apache/flink/pull/25717#discussion_r1898405938 ## flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/rank/asyncprocessing/AbstractAsyncSyncStateTopNFunction.java: ## @@ -0,0 +

Re: [PR] [hotfix] Allow pinning issues & PRs that never decay [flink-cdc]

2024-12-27 Thread via GitHub
yuxiqian commented on PR #3816: URL: https://github.com/apache/flink-cdc/pull/3816#issuecomment-2563527552 @leonardBang This will work by attaching a "pinned" label and remove "Stale" label from an issue or PR. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [hotfix] Allow pinning issues & PRs that never decay [flink-cdc]

2024-12-27 Thread via GitHub
leonardBang merged PR #3816: URL: https://github.com/apache/flink-cdc/pull/3816 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-36576][runtime] Improving amount-based data balancing distribution algorithm for DefaultVertexParallelismAndInputInfosDecider [flink]

2024-12-27 Thread via GitHub
JunRuiLee commented on code in PR #25552: URL: https://github.com/apache/flink/pull/25552#discussion_r1898365716 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptivebatch/util/SubpartitionSlice.java: ## @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Softw

[jira] [Updated] (FLINK-36970) Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING

2024-12-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36970: --- Labels: pull-request-available (was: ) > Merge result of data type BIGINT and DOUBLE should

[PR] [FLINK-36970][runtime] merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING [flink-cdc]

2024-12-27 Thread via GitHub
Shawn-Hx opened a new pull request, #3821: URL: https://github.com/apache/flink-cdc/pull/3821 In SchemaMergingUtils#getLeastCommonType, the merge result of BIGINT and DOUBLE is STRING now. However, considering JSON format, JSON numbers can be integers or floating point, which can be

Re: [PR] [FLINK-36576][runtime] Improving amount-based data balancing distribution algorithm for DefaultVertexParallelismAndInputInfosDecider [flink]

2024-12-27 Thread via GitHub
JunRuiLee commented on code in PR #25552: URL: https://github.com/apache/flink/pull/25552#discussion_r1898349498 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptivebatch/BlockingInputInfo.java: ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software F

Re: [PR] [hotfix][postgres] Flink CDC Postgres Connector subsribe regular expr… [flink-cdc]

2024-12-27 Thread via GitHub
hql0312 commented on code in PR #3808: URL: https://github.com/apache/flink-cdc/pull/3808#discussion_r1898300731 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/main/java/org/apache/flink/cdc/connectors/postgres/source/utils/CustomPostgresSchema.java: #

[jira] [Updated] (FLINK-36970) Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING

2024-12-27 Thread Xiao Huang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Huang updated FLINK-36970: --- Description: In SchemaMergingUtils#getLeastCommonType, the merge result of BIGINT and DOUBLE is STR

[jira] [Created] (FLINK-36970) Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING

2024-12-27 Thread Xiao Huang (Jira)
Xiao Huang created FLINK-36970: -- Summary: Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING Key: FLINK-36970 URL: https://issues.apache.org/jira/browse/FLINK-36970 Project: F

[jira] [Created] (FLINK-36969) Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING

2024-12-27 Thread Xiao Huang (Jira)
Xiao Huang created FLINK-36969: -- Summary: Merge result of data type BIGINT and DOUBLE should be DOUBLE instead of STRING Key: FLINK-36969 URL: https://issues.apache.org/jira/browse/FLINK-36969 Project: F

Re: [PR] [FLINK-36576][runtime] Improving amount-based data balancing distribution algorithm for DefaultVertexParallelismAndInputInfosDecider [flink]

2024-12-27 Thread via GitHub
JunRuiLee commented on code in PR #25552: URL: https://github.com/apache/flink/pull/25552#discussion_r1898349498 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptivebatch/BlockingInputInfo.java: ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software F

[jira] [Updated] (FLINK-36960) FLIP-467 Introduce Generalized Watermarks for DataStream V2

2024-12-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36960: --- Labels: pull-request-available (was: ) > FLIP-467 Introduce Generalized Watermarks for Data

Re: [PR] [FLINK-36960] Introduce Generalized Watermark for DataStream V2 [flink]

2024-12-27 Thread via GitHub
reswqa commented on code in PR #25731: URL: https://github.com/apache/flink/pull/25731#discussion_r1898186510 ## flink-core-api/src/main/java/org/apache/flink/api/common/watermark/WatermarkCombinationFunction.java: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] [FLINK-36576][runtime] Improving amount-based data balancing distribution algorithm for DefaultVertexParallelismAndInputInfosDecider [flink]

2024-12-27 Thread via GitHub
JunRuiLee commented on code in PR #25552: URL: https://github.com/apache/flink/pull/25552#discussion_r1898321168 ## flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/JobVertex.java: ## @@ -522,11 +522,42 @@ public JobEdge connectNewDataSetAsInput( Interm