Re: [DISCUSS] FLIP-307: Flink connector Redshift

2023-08-28 Thread Samrat Deb
Hi Martijn, I fully comprehend your apprehensions regarding AWS's reliance on JDBC and the resultant impact on the release cycle. However, an effective approach is to utilize the latest version of flink-connector-jdbc as a Maven dependency. In this arrangement, flink-connector-aws can be associated

Re: [DISCUSS][2.0] Deprecating Accumulator in favor of MetricsGroup

2023-08-28 Thread David Morávek
AFAIK Apache Beam also used acummulators for metric collection, which is indeed a major use case. I’m not convinced that MetricGroup is fuĺly replacing what acummulators have to offer though; OperatorCoordinators might be able to rplace remaining capabilities, but this need bit more thoughts, the

Re: [DISCUSS][2.0] Deprecating Accumulator in favor of MetricsGroup

2023-08-28 Thread Xintong Song
Thanks for bringing this up, Matthias. One thing that a user may achieve with an accumulator but not with a metric group is to programmatically fetch the job execution result, rather than outputting the results to an external sink, in attached mode. This can also be achieved by using CollectSink,

[jira] [Created] (FLINK-32986) The new createTemporaryFunction has some regression of type inference compare to the deprecated registerFunction

2023-08-28 Thread lincoln lee (Jira)
lincoln lee created FLINK-32986: --- Summary: The new createTemporaryFunction has some regression of type inference compare to the deprecated registerFunction Key: FLINK-32986 URL: https://issues.apache.org/jira/browse

[DISCUSS] FLIP-358: flink-avro enhancement and cleanup

2023-08-28 Thread Becket Qin
Hi folks, I would like to start the discussion about FLIP-158[1] which proposes to clean up and enhance the Avro support in Flink. More specifically, it proposes to: 1. Make it clear what are the public APIs in flink-avro components. 2. Fix a few buggy cases in flink-avro 3. Add more supported Av

Re: log4j2 integration with flink 1.10

2023-08-28 Thread David Anderson
Flink switched to using log4j2 with the 1.11 release. Looking at the ticket involved [1] should give you some idea of the effort involved in backporting that to 1.10. My initial impression is that you might not have to do much to get it working, The comments in [2] appear to outline what's needed.

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-28 Thread Alexey Leonov-Vendrovskiy
SGTM For future reference, responding here: 5) Any operator can introduce system (psedo) columns. > > This is clearly out of scope for this FLIP. The implementation effort > would be huge and could introduce a lot of bugs. > I didn't imply any specific implementation or feature coverage in the

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-28 Thread Venkatakrishnan Sowrirajan
Hi Jinsong, Thanks for your comments. What is inputIndex in NestedFieldReferenceExpression? I haven't looked at it before. Do you mean, given that it is now only used to push filters it won't be subsequently used in further planning/optimization and therefore it is not required at this time? S

[jira] [Created] (FLINK-32985) Support setting env.java.opts.sql-gateway to specify the jvm opts for sql gateway

2023-08-28 Thread xiangyu feng (Jira)
xiangyu feng created FLINK-32985: Summary: Support setting env.java.opts.sql-gateway to specify the jvm opts for sql gateway Key: FLINK-32985 URL: https://issues.apache.org/jira/browse/FLINK-32985 Pro

[jira] [Created] (FLINK-32984) Display the host port information on the suttasks page of SubtaskExecution

2023-08-28 Thread Matt Wang (Jira)
Matt Wang created FLINK-32984: - Summary: Display the host port information on the suttasks page of SubtaskExecution Key: FLINK-32984 URL: https://issues.apache.org/jira/browse/FLINK-32984 Project: Flink

[jira] [Created] (FLINK-32983) Support setting env.java.opts.all & env.java.opts.cli configs via dynamic properties on the CLI side

2023-08-28 Thread Zhanghao Chen (Jira)
Zhanghao Chen created FLINK-32983: - Summary: Support setting env.java.opts.all & env.java.opts.cli configs via dynamic properties on the CLI side Key: FLINK-32983 URL: https://issues.apache.org/jira/browse/FLINK-3

[jira] [Created] (FLINK-32982) Don't need to suggest to swap for using Hive dialect

2023-08-28 Thread luoyuxia (Jira)
luoyuxia created FLINK-32982: Summary: Don't need to suggest to swap for using Hive dialect Key: FLINK-32982 URL: https://issues.apache.org/jira/browse/FLINK-32982 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-32981) Add python dynamic Flink home detection

2023-08-28 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-32981: - Summary: Add python dynamic Flink home detection Key: FLINK-32981 URL: https://issues.apache.org/jira/browse/FLINK-32981 Project: Flink Issue Type: Improve

[jira] [Created] (FLINK-32980) Support env.java.opts.all & env.java.opts.cli config for starting Session clusters

2023-08-28 Thread Zhanghao Chen (Jira)
Zhanghao Chen created FLINK-32980: - Summary: Support env.java.opts.all & env.java.opts.cli config for starting Session clusters Key: FLINK-32980 URL: https://issues.apache.org/jira/browse/FLINK-32980

[jira] [Created] (FLINK-32979) Deprecate WindowAssigner#getDefaultTrigger(StreamExecutionEnvironment env)

2023-08-28 Thread Wencong Liu (Jira)
Wencong Liu created FLINK-32979: --- Summary: Deprecate WindowAssigner#getDefaultTrigger(StreamExecutionEnvironment env) Key: FLINK-32979 URL: https://issues.apache.org/jira/browse/FLINK-32979 Project: Fli

[jira] [Created] (FLINK-32978) Deprecate RichFunction#open(Configuration parameters)

2023-08-28 Thread Wencong Liu (Jira)
Wencong Liu created FLINK-32978: --- Summary: Deprecate RichFunction#open(Configuration parameters) Key: FLINK-32978 URL: https://issues.apache.org/jira/browse/FLINK-32978 Project: Flink Issue Typ

[jira] [Created] (FLINK-32976) NullpointException when starting flink cluster

2023-08-28 Thread Feng Jin (Jira)
Feng Jin created FLINK-32976: Summary: NullpointException when starting flink cluster Key: FLINK-32976 URL: https://issues.apache.org/jira/browse/FLINK-32976 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-32975) Enhance equal() for ChangelogMapState's iterator

2023-08-28 Thread Rui Xia (Jira)
Rui Xia created FLINK-32975: --- Summary: Enhance equal() for ChangelogMapState's iterator Key: FLINK-32975 URL: https://issues.apache.org/jira/browse/FLINK-32975 Project: Flink Issue Type: Improvemen

[jira] [Created] (FLINK-32974) RestClusterClient leaks flink-rest-client-jobgraphs* directories

2023-08-28 Thread Lijie Wang (Jira)
Lijie Wang created FLINK-32974: -- Summary: RestClusterClient leaks flink-rest-client-jobgraphs* directories Key: FLINK-32974 URL: https://issues.apache.org/jira/browse/FLINK-32974 Project: Flink

[jira] [Created] (FLINK-32973) Documentation for kubernetes.operator.job.savepoint-on-deletion indicate that it can be set on per resource level while it can't

2023-08-28 Thread Nicolas Fraison (Jira)
Nicolas Fraison created FLINK-32973: --- Summary: Documentation for kubernetes.operator.job.savepoint-on-deletion indicate that it can be set on per resource level while it can't Key: FLINK-32973 URL: https://issu

[jira] [Created] (FLINK-32972) TaskTest.testInterruptibleSharedLockInInvokeAndCancel causes a JVM shutdown with exit code 239

2023-08-28 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-32972: --- Summary: TaskTest.testInterruptibleSharedLockInInvokeAndCancel causes a JVM shutdown with exit code 239 Key: FLINK-32972 URL: https://issues.apache.org/jira/browse/FLINK-329

[jira] [Created] (FLINK-32971) Add pyflink proper development version support

2023-08-28 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-32971: - Summary: Add pyflink proper development version support Key: FLINK-32971 URL: https://issues.apache.org/jira/browse/FLINK-32971 Project: Flink Issue Type:

Re: [Re-DISCUSS] FLIP-202: Introduce ClickHouse Connector

2023-08-28 Thread ConradJam
RocMarshal mentioned the new connector API in the FLINK-26999 [1] , I wonder if this refers to Sink V2 Api [2] Or other things? If someone can help me troubleshoot this issue it will speed up my design progress - [1] FLINK-26999: https://issues.apache.org/jira/browse/FLINK-26999 - [2] Sink

[jira] [Created] (FLINK-32970) could not load external class using "-C" option

2023-08-28 Thread Spongebob (Jira)
Spongebob created FLINK-32970: - Summary: could not load external class using "-C" option Key: FLINK-32970 URL: https://issues.apache.org/jira/browse/FLINK-32970 Project: Flink Issue Type: Bug