[jira] [Created] (FLINK-27081) Remove Azure integration

2022-04-05 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27081: Summary: Remove Azure integration Key: FLINK-27081 URL: https://issues.apache.org/jira/browse/FLINK-27081 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27080) Enable builds for pull requests

2022-04-05 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27080: Summary: Enable builds for pull requests Key: FLINK-27080 URL: https://issues.apache.org/jira/browse/FLINK-27080 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27079) Setup cron jobs

2022-04-05 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27079: Summary: Setup cron jobs Key: FLINK-27079 URL: https://issues.apache.org/jira/browse/FLINK-27079 Project: Flink Issue Type: Sub-task Compon

[jira] [Created] (FLINK-27077) Setup per-commit builds for master

2022-04-05 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27077: Summary: Setup per-commit builds for master Key: FLINK-27077 URL: https://issues.apache.org/jira/browse/FLINK-27077 Project: Flink Issue Type: Sub-ta

[jira] [Created] (FLINK-27078) There is a performance gap between the new csv source(file system source + CSV format) and legacy CsvTableSource.

2022-04-05 Thread Lijie Wang (Jira)
Lijie Wang created FLINK-27078: -- Summary: There is a performance gap between the new csv source(file system source + CSV format) and legacy CsvTableSource. Key: FLINK-27078 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (FLINK-27076) Setup runners

2022-04-05 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27076: Summary: Setup runners Key: FLINK-27076 URL: https://issues.apache.org/jira/browse/FLINK-27076 Project: Flink Issue Type: Sub-task Componen

[jira] [Created] (FLINK-27075) Migrate CI from Azure to Github Actions

2022-04-05 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27075: Summary: Migrate CI from Azure to Github Actions Key: FLINK-27075 URL: https://issues.apache.org/jira/browse/FLINK-27075 Project: Flink Issue Type: T

[jira] [Created] (FLINK-27074) python_job.py should write to TEST_DATA_DIR

2022-04-05 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27074: Summary: python_job.py should write to TEST_DATA_DIR Key: FLINK-27074 URL: https://issues.apache.org/jira/browse/FLINK-27074 Project: Flink Issue Typ

[jira] [Created] (FLINK-27073) HAQueryableStateFsBackendITCase hangs on azure

2022-04-05 Thread Yun Gao (Jira)
Yun Gao created FLINK-27073: --- Summary: HAQueryableStateFsBackendITCase hangs on azure Key: FLINK-27073 URL: https://issues.apache.org/jira/browse/FLINK-27073 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-27072) Add Bucketizer in FlinkML

2022-04-05 Thread Zhipeng Zhang (Jira)
Zhipeng Zhang created FLINK-27072: - Summary: Add Bucketizer in FlinkML Key: FLINK-27072 URL: https://issues.apache.org/jira/browse/FLINK-27072 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-27071) For jdbcsink function to support config table name in SQL parameter

2022-04-05 Thread haixiaCao (Jira)
haixiaCao created FLINK-27071: - Summary: For jdbcsink function to support config table name in SQL parameter Key: FLINK-27071 URL: https://issues.apache.org/jira/browse/FLINK-27071 Project: Flink

[jira] [Created] (FLINK-27070) Reuse FileFormat instead of DecodingFormat/EncodingFormat to ensure thread safety.

2022-04-05 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-27070: --- Summary: Reuse FileFormat instead of DecodingFormat/EncodingFormat to ensure thread safety. Key: FLINK-27070 URL: https://issues.apache.org/jira/browse/FLINK-27070 Proj

Re: pyflink on Python 3.9 dev guide

2022-04-05 Thread Luan Cooper
actually I had build/compile - pyarrow==2.0.0 (test skipped) - apache-beam==2.27.0 (test skipped) on python 3.9, and test with example python jobs( bin/flink run -pyclientexec python3.7 -pyexec python3.9 -py examples/python/table/word_count.py ) but got exceptions following Caused by: java.util.co

[jira] [Created] (FLINK-27069) Fix the potential memory corruption in Thread Mode

2022-04-05 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-27069: Summary: Fix the potential memory corruption in Thread Mode Key: FLINK-27069 URL: https://issues.apache.org/jira/browse/FLINK-27069 Project: Flink Issue Type

[jira] [Created] (FLINK-27068) test_keyed_min_and_max and test_keyed_min_by_and_max_by failed in py36,37

2022-04-05 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-27068: Summary: test_keyed_min_and_max and test_keyed_min_by_and_max_by failed in py36,37 Key: FLINK-27068 URL: https://issues.apache.org/jira/browse/FLINK-27068 Project: Fl

Re: Creating flink-kubernetes-operator project on Dockerhub

2022-04-05 Thread Yang Wang
It seems that you have already got this done :) One more question, do we need to create an official docker image repo for flink-kubernetes-operator[1]? Then user could pull the image directly via "docker pull flink-kubernetes-operator". The drawback is we always need to create a PR[2] and wait for

[jira] [Created] (FLINK-27067) Prevent usage of deprecated APIs in PyFlink examples

2022-04-05 Thread Dian Fu (Jira)
Dian Fu created FLINK-27067: --- Summary: Prevent usage of deprecated APIs in PyFlink examples Key: FLINK-27067 URL: https://issues.apache.org/jira/browse/FLINK-27067 Project: Flink Issue Type: Improv

Re: [DISCUSS] Flink SQL Syntax for Query/Savepoint Management

2022-04-05 Thread Paul Lam
Hi Timo, Thanks for you reply! > It would be great to further investigate which other commands are required > that would be usually be exeuted via CLI commands. I would like to avoid a > large amount of FLIPs each adding a special job lifecycle command. Okay. I listed only the commands about j

Re: [ANNOUNCE] Apache Flink Kubernetes Operator 0.1.0 released

2022-04-05 Thread Yang Wang
Thanks Gyula for the great work. It is a big step for making Flink more cloud-native. Best, Yang Xintong Song 于2022年4月6日周三 10:37写道: > Thanks Gyula for driving this, and everyone who helped make this release > possible. > > Kindly reminder: It seems we have not added this release to the ASF rep

Re: [DISCUSS] Bumping operator CRD version to v1alpha2

2022-04-05 Thread Yang Wang
I also lean to the second option since v1alpha1 is a preview release. But we need to be more careful to introduce other incompatible changes after v1beta1. Maybe we also need a simple manual for how to upgrade the operator especially when the CRD version changed. Best, Yang Gyula Fóra 于2022年4月5

Re: [ANNOUNCE] Apache Flink Kubernetes Operator 0.1.0 released

2022-04-05 Thread Xintong Song
Thanks Gyula for driving this, and everyone who helped make this release possible. Kindly reminder: It seems we have not added this release to the ASF report database. See "Recordkeeping" in the release process.[1] Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/C

Re: Regarding java.lang.NullPointerException

2022-04-05 Thread Xingbo Huang
Hi Josemon, Can you provide the version of PyFlink you are using, and it would be better if you can also provide code snippets that can be easily reproduced this problem. Best, Xingbo Josemon Maliakal 于2022年4月5日周二 13:31写道: > Hello There, > I started using pyflink recently, > Have anyone know w

Re: pyflink on Python 3.9 dev guide

2022-04-05 Thread Xingbo Huang
Hi Martjin and Luan, As of now, the main reason why PyFlink has not declared to support Python 3.9 is that the dependent apache-beam, and the versions of numpy and pyarrow that apache-beam depends on do not provide corresponding whl packages in Python 3.9. Users need source code installation, but

Re: Question about community collaboration options

2022-04-05 Thread Hao t Chang
Hi Martin, Thanks for the clarification. I will be interested in becoming an ASF committer. I missed the Sync meeting today because I thought 4pm CST was Central Standard Time but it was actually my 1am(Pacific Time). Is there no contributor joining from the US ? I will join the next week sync f

[jira] [Created] (FLINK-27066) Reintroduce e2e tests in ES as Java tests.

2022-04-05 Thread Alexander Fedulov (Jira)
Alexander Fedulov created FLINK-27066: - Summary: Reintroduce e2e tests in ES as Java tests. Key: FLINK-27066 URL: https://issues.apache.org/jira/browse/FLINK-27066 Project: Flink Issue Ty

[jira] [Created] (FLINK-27064) Centralize ArchUnit rules for production code

2022-04-05 Thread Jing Ge (Jira)
Jing Ge created FLINK-27064: --- Summary: Centralize ArchUnit rules for production code Key: FLINK-27064 URL: https://issues.apache.org/jira/browse/FLINK-27064 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-27065) Store lastReconciledSpec as String in status

2022-04-05 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-27065: -- Summary: Store lastReconciledSpec as String in status Key: FLINK-27065 URL: https://issues.apache.org/jira/browse/FLINK-27065 Project: Flink Issue Type: Improvem

[jira] [Created] (FLINK-27063) Upgrade Hive 2.3 connector to version 2.3.6

2022-04-05 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-27063: -- Summary: Upgrade Hive 2.3 connector to version 2.3.6 Key: FLINK-27063 URL: https://issues.apache.org/jira/browse/FLINK-27063 Project: Flink Issue Type: T

[jira] [Created] (FLINK-27062) Add Boring Cyborg bot to flink-connector-elasticsearch

2022-04-05 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-27062: -- Summary: Add Boring Cyborg bot to flink-connector-elasticsearch Key: FLINK-27062 URL: https://issues.apache.org/jira/browse/FLINK-27062 Project: Flink Is

[jira] [Created] (FLINK-27061) Improve the ArchUnit test infra in the external elasticsearch connector

2022-04-05 Thread Jing Ge (Jira)
Jing Ge created FLINK-27061: --- Summary: Improve the ArchUnit test infra in the external elasticsearch connector Key: FLINK-27061 URL: https://issues.apache.org/jira/browse/FLINK-27061 Project: Flink

[jira] [Created] (FLINK-27060) Extending /jars/:jarid/run API to

2022-04-05 Thread Zhanghao Chen (Jira)
Zhanghao Chen created FLINK-27060: - Summary: Extending /jars/:jarid/run API to Key: FLINK-27060 URL: https://issues.apache.org/jira/browse/FLINK-27060 Project: Flink Issue Type: Improvement

[ANNOUNCE] Dependabot has been enabled for the Apache organization including the Flink project

2022-04-05 Thread Martijn Visser
Hi everyone, As part of our efforts to externalize connectors, I've also been in contact with the ASF to ask if it's possible to enable the Dependabot functionality of Github. It was previously possible to only enable the notification feature of Dependabot (which notifies you if there's a security

[jira] [Created] (FLINK-27059) [JUnit5 Migration] Module: flink-compress

2022-04-05 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-27059: --- Summary: [JUnit5 Migration] Module: flink-compress Key: FLINK-27059 URL: https://issues.apache.org/jira/browse/FLINK-27059 Project: Flink Issue Type: Sub-task

Re: pyflink on Python 3.9 dev guide

2022-04-05 Thread Martijn Visser
Hi Luan, According to the documentation Python 3.9 is currently indeed not supported. I briefly checked the Jira tickets and also couldn't find one about adding support for this, so I've created https://issues.apache.org/jira/browse/FLINK-27058 for that. @dian0511...@gmail.com @hxbks...@gmail.co

[jira] [Created] (FLINK-27058) Add support for Python 3.9

2022-04-05 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-27058: -- Summary: Add support for Python 3.9 Key: FLINK-27058 URL: https://issues.apache.org/jira/browse/FLINK-27058 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-27057) Connector to external repo migration - Elasticsearch Connector

2022-04-05 Thread Jing Ge (Jira)
Jing Ge created FLINK-27057: --- Summary: Connector to external repo migration - Elasticsearch Connector Key: FLINK-27057 URL: https://issues.apache.org/jira/browse/FLINK-27057 Project: Flink Issue T

Re: Flink 1.15 Stabilisation Sync

2022-04-05 Thread Yuan Mei
FLINK-26985 was discovered just before last weekend. We will get it resolved first thing after the holiday (tomorrow). Best Yuan On Tue, Apr 5, 2022 at 5:37 PM Yun Gao wrote: > Hi Robert, > > Very sorry for the long delay before the rc1 could be published. > > For the open critical issues, I p

Re: Pyflink + TABLE API + snowflake

2022-04-05 Thread Shameet Doshi
I see Would that mean to contribute to the flink connector source code or I can create it as part of another dependant jar On Tue, Apr 5, 2022 at 5:36 AM Martijn Visser wrote: > Hi Shameet, > > No, you'll need to add support for the Snowflake dialect in Flink. You can > find more information h

pyflink on Python 3.9 dev guide

2022-04-05 Thread Luan Cooper
Hi currently I'll need to run pyflink udf on python 3.9 which is not supported right now I tried building - pyarrow==2.0.0 - apache-beam==2.27.0 on python 3.9 and test python jobs but failed Is there any discussions/git branch on python 3.9 before? (I didn't find any in this dev list) so I can c

Re: Flink 1.15 Stabilisation Sync

2022-04-05 Thread Yun Gao
Hi Robert, Very sorry for the long delay before the rc1 could be published. For the open critical issues, I previously checked with the owners of the issues that they are optional to the release, thus the only blocker issue is the FLINK-26985. Besides to the blocker issues we are waiting for

Re: Pyflink + TABLE API + snowflake

2022-04-05 Thread Martijn Visser
Hi Shameet, No, you'll need to add support for the Snowflake dialect in Flink. You can find more information here at the JdbcDialect documentation [1] Best regards, Martijn Visser https://twitter.com/MartijnVisser82 https://github.com/MartijnVisser [1] https://nightlies.apache.org/flink/flink-d

Re: Pyflink + TABLE API + snowflake

2022-04-05 Thread Shameet Doshi
thanks Martijn So supplying the snowflake-jdbc jar as a dependency jar as i have done and mentioning the driver property doesn't help ? t_env.get_config().get_configuration().set_string("pipeline.jars", "file:///Users/shameetdoshi/Downloads/flink-connector-jdbc_2.12-1.14.4.jar;file:///Users/shame

Re: Pyflink + TABLE API + snowflake

2022-04-05 Thread Martijn Visser
Hi Shameet, There's currently no open source Flink Snowflake connector/sink available. As mentioned in the documentation [1] this requires the implementation of a dialect. Best regards Martijn Visser https://twitter.com/MartijnVisser82 https://github.com/MartijnVisser [1] https://nightlies.apac