Re: [ANNOUNCE] Apache Flink 1.10.2 released

Thanks Zhu for the great work and everyone who contributed to this release! Best, Xingbo Guowei Ma 于2020年8月26日周三 下午12:43写道： > Hi, > > Thanks a lot for being the release manager Zhu Zhu! > Thanks everyone contributed to this! > > Best, > Guowei > > > On Wed, Aug 26, 2020 at 11:18 AM Yun Tang wr

Re: [ANNOUNCE] Apache Flink 1.10.2 released

Hi, Thanks a lot for being the release manager Zhu Zhu! Thanks everyone contributed to this! Best, Guowei On Wed, Aug 26, 2020 at 11:18 AM Yun Tang wrote: > Thanks for Zhu's work to manage this release and everyone who contributed > to this! > > Best, > Yun Tang >

[DISCUSS] FLIP-139: General Python User-Defined Aggregate Function on Table API

Hi everyone, I would like to start discussion about how to support General Python User-Defined Aggregate Function on Table API. FLIP-58[1] has already introduced the stateless Python UDF and has already been supported in the previous releases. However the stateful Python UDF, i.e. User-Defined

Re: [ANNOUNCE] Apache Flink 1.10.2 released

Thanks for Zhu's work to manage this release and everyone who contributed to this! Best, Yun Tang From: Yangze Guo Sent: Tuesday, August 25, 2020 14:47 To: Dian Fu Cc: Zhu Zhu ; dev ; user ; user-zh Subject: Re: [ANNOUNCE] Apache Flink 1.10.2 released Thanks

HadoopOutputFormat has issues with LocalExecutionEnvironment?

Hi devs, In HadoopOutputFormat.close(), I see code that is trying to rename /tmp-r-1 to be /1 But when I run my Flink 1.9.2 code using a local MiniCluster, the actual location of the tmp-r-1 file is: /_temporary/0/task___r_01/tmp-r-1 I think this is because the default beh

Re: [DISCUSS] Remove Kafka 0.10.x connector (and possibly 0.11.x)

+1 to remove both the 1.10 and 1.11 connectors. The connectors have not been actively developed for some time. They are basically just sitting around causing noise by causing test instabilities and eating CI time. It would  also allow us to really simplify the module structure of the Kafka con

[ANNOUNCE] Weekly Community Update 2020/31-34

Dear community, The "weekly" community update is back after a short summer break! This time I've tried to cover most of what happened during the last four weeks, but I might pick up some older topics in the next weeks' updates, too. Activity on the dev@ mailing list has picked up quite a bit as f

Re: [DISCUSS] Removing deprecated methods from DataStream API

I would argue that the guarantees of @Public methods that became ineffective were broken when they became ineffective (and were deprecated). - ExecutionConfig#disable/enableSysoutLogging (deprecated in 1.10) - ExecutionConfig#set/isFailTaskOnCheckpointError (deprecated in 1.9) Removing thes

Re: [DISCUSS] Remove Kafka 0.10.x connector (and possibly 0.11.x)

Hi Aljoscha, I am assuming you're asking about dropping the flink-connector-kafka-0.10/0.11 modules, right? Or are you talking about removing support for Kafka 0.10/0.11 from the universal connector? I am in favor of removing flink-connector-kafka-0.10/0.11 in the next release. These modules woul

Re: How jobmanager and task manager communicates with each other ?

Hi Sidhant, (1) If we are not using Flink's HA services then how we can dynamically > configure task manager nodes to connect to job manager? Any suggestions or > best practices? Not sure what you mean by 'dynamically'. I think you have to restart the task manager with the new configuration to co

Re: How jobmanager and task manager communicates with each other ?

Hi Till, Thanks for the reply. (1) If we are not using Flink's HA services then how we can dynamically configure task manager nodes to connect to job manager? Any suggestions or best practices? (2) Which and how flink's HA service can be used for the service discovery of job manager ? Regards S

Re: [ANNOUNCE] New Flink Committer: David Anderson

Congrats, David. Well deserved. On Thu, Aug 20, 2020 at 5:06 AM Matt Wang wrote: > Congrats David! > > > -- > > Best, > Matt Wang > > > On 08/20/2020 10:44，Zhijiang wrote： > Congratulations David! > > > -- > From:Jeff Zhang > Send

[jira] [Created] (FLINK-19050) Doc of MAX_DECIMAL_PRECISION should be DECIMAL

Pua created FLINK-19050: --- Summary: Doc of MAX_DECIMAL_PRECISION should be DECIMAL Key: FLINK-19050 URL: https://issues.apache.org/jira/browse/FLINK-19050 Project: Flink Issue Type: Bug Compon

Re: Next Stateful Functions Release

Igal, The feature set you propose sounds great to me -- as a user I see plenty there to get excited about. As for the feature freeze date, I don't really have an informed opinion. David On Mon, Aug 24, 2020 at 10:15 AM Igal Shilman wrote: > > Hi Flink devs, > > We have a few upcoming / implemen

[jira] [Created] (FLINK-19049) TableEnvironmentImpl.executeInternal() does not wait for the final job status

Robert Metzger created FLINK-19049: -- Summary: TableEnvironmentImpl.executeInternal() does not wait for the final job status Key: FLINK-19049 URL: https://issues.apache.org/jira/browse/FLINK-19049 Pro

Re: [DISCUSS] FLIP-36 - Support Interactive Programming in Flink Table API

Hi Timo, Thanks for your comments. After the offline discussion, I have updated the FLIP with the following change. 1. Update the end to end process a. The Table.cache method should only wrap the origin query operation with CacheOperation. b. The planner will add the CacheSink or replac

Re: [flink-sql-connector-elasticsearch]相关问题

Hi all, I'd like to kindly remind you that dev@flink.apache.org is an international mailing list. It is important to have the discussions in English, so that every subscriber can understand. Discussions in Chinese can be posted in the user...@flink.apache.org mailing list. Please refer to the comm

[jira] [Created] (FLINK-19048) suuport line format for table connector

badqiu created FLINK-19048: -- Summary: suuport line format for table connector Key: FLINK-19048 URL: https://issues.apache.org/jira/browse/FLINK-19048 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-19047) Move unaligned checkpoint methods from ResultPartition to separate interface.

Stephan Ewen created FLINK-19047: Summary: Move unaligned checkpoint methods from ResultPartition to separate interface. Key: FLINK-19047 URL: https://issues.apache.org/jira/browse/FLINK-19047 Project

Re: [flink-sql-connector-elasticsearch]相关问题

pom里面把这个依赖设置为provided,然后将这个jar包copy到服务器上flink的lib中，然后打包项目，在运行，试一下。 在 2020/8/25 17:38, Li,Qian(DXM,PB) 写道: Hi,all: 我在使用Flink SQL CLI向ES6写数据的时候，任务一直执行失败， Log日志显示没有ElasticsearchSink类，请问是什么原因造成的呢？ 我是用的jar包是这个flink-sql-connector-elasticsearch6_2.11-1.11.0.jar，ES版本是6.5。 谢谢～ 2020-08-25 17:19:38,245 W

[jira] [Created] (FLINK-19046) Introduce separate classes for PipelinedResultPartition and BoundedBlockingResultPartition

Stephan Ewen created FLINK-19046: Summary: Introduce separate classes for PipelinedResultPartition and BoundedBlockingResultPartition Key: FLINK-19046 URL: https://issues.apache.org/jira/browse/FLINK-19046

[jira] [Created] (FLINK-19045) Remove obsolete option 'taskmanager.network.partition.force-release-on-consumption'

Stephan Ewen created FLINK-19045: Summary: Remove obsolete option 'taskmanager.network.partition.force-release-on-consumption' Key: FLINK-19045 URL: https://issues.apache.org/jira/browse/FLINK-19045 P

[flink-sql-connector-elasticsearch]相关问题

Hi,all: 我在使用Flink SQL CLI向ES6写数据的时候，任务一直执行失败， Log日志显示没有ElasticsearchSink类，请问是什么原因造成的呢？ 我是用的jar包是这个flink-sql-connector-elasticsearch6_2.11-1.11.0.jar，ES版本是6.5。 谢谢～ 2020-08-25 17:19:38,245 WARN org.apache.flink.runtime.taskmanager.Task [] - Source: TableSourceScan(table=[[default_catalog, defaul

Re: [DISCUSS] Whether catalog table factory should be used for temporary tables

Hi, Thanks everyone for your inputs. Temporary hive table is not supported at the moment. If we want to support it, I agree with Jingsong that the life cycle of the temporary table should somehow be bound to the hive catalog. For example, hive catalog should be responsible to delete the table fol

Re: [DISCUSS] Whether catalog table factory should be used for temporary tables

Hi Jark, You raised a good point: Creating the Hive temporary table. AFAIK, Hive temporary tables should be stored in metastore, Hive metastore will maintain their life cycle. Correct me if I am wrong. So actually, if we want to support Hive temporary tables, we should finish one thing: - A tempo

[DISCUSS] FLIP-138: Declarative Resource management

Hello, in FLIP-138 we want to rework the way the JobMaster acquires slots, such that required resources are declared before a job is scheduled and th job execution is adjusted according to the provided resources (e.g., reducing parallelism), instead of asking for a fixed number of resources d

Re: [DISCUSS] Whether catalog table factory should be used for temporary tables

Hi, I'm wondering if we always fallback to using SPI for temporary tables, then how does the create Hive temporary table using Hive dialect work? IMO, adding an "isTemporary" to the factory context sounds reasonable to me, because the factory context should describe the full content of create tab

[jira] [Created] (FLINK-19044) Web UI reports incorrect timeline for the FINISHED state of a subtask

Caizhi Weng created FLINK-19044: --- Summary: Web UI reports incorrect timeline for the FINISHED state of a subtask Key: FLINK-19044 URL: https://issues.apache.org/jira/browse/FLINK-19044 Project: Flink

Re: [DISCUSS] Whether catalog table factory should be used for temporary tables

Hi Dawid, But the temporary table does not belong to Catalog, actually Catalog doesn't know the existence of the temporary table. Let the table factory of catalog to create source/sink sounds a little sudden. If we want to make temporary tables belong to Catalog, I think we need to involve catalo

[jira] [Created] (FLINK-19043) Translate the page 'Logging' of 'Debugging & Monitoring' into Chinese

Roc Marshal created FLINK-19043: --- Summary: Translate the page 'Logging' of 'Debugging & Monitoring' into Chinese Key: FLINK-19043 URL: https://issues.apache.org/jira/browse/FLINK-19043 Project: Flink

Re: [DISCUSS] Whether catalog table factory should be used for temporary tables

Hi Rui, My take is that temporary tables should use the factory of the catalog they were registered with. What you are describing sounds very much like a limitation/bug in Hive catalog only. I'd be in favor of passing the *isTemporary* flag. Best, Dawid On 25/08/2020 09:37, Rui Li wrote: > Hi

[DISCUSS] Whether catalog table factory should be used for temporary tables

Hi Dev, Currently temporary generic tables cannot work with hive catalog [1]. When hive catalog is chosen as the current catalog, planner will use HiveTableFactory to create source/sink for the temporary table. HiveTableFactory cannot tell whether a table is temporary or not, and considers it as a

[jira] [Created] (FLINK-19042) HiveTableSourceITCase fails if object reuse is enabled

Rui Li created FLINK-19042: -- Summary: HiveTableSourceITCase fails if object reuse is enabled Key: FLINK-19042 URL: https://issues.apache.org/jira/browse/FLINK-19042 Project: Flink Issue Type: Test

Re: [DISCUSS][Release 1.12] Stale blockers and build instabilities

Thanks Dian for the pointer. I'll take a look. On Tue, Aug 25, 2020 at 3:02 PM Dian Fu wrote: > Thanks Rui for the info. This issue(hive related) > https://issues.apache.org/jira/browse/FLINK-19025 < > https://issues.apache.org/jira/browse/FLINK-19025> is marked as a blocker. > > Regards, > Dian

Re: [DISCUSS][Release 1.12] Stale blockers and build instabilities

Thanks Rui for the info. This issue(hive related) https://issues.apache.org/jira/browse/FLINK-19025 is marked as a blocker. Regards, Dian > 在 2020年8月25日，下午2:58，Rui Li 写道： > > Hi Dian, > > FLINK-18682 has been fixed. Is there any other block