taskexecutor .out files

2022-05-15 Thread Zain Haider Nemati
Hi, I have been running a streaming job which prints data to .out files the size of the file has gotten really large and is choking the root memory for my VM. Is it ok to delete the .out files? Would that affect any other operation or functionality?

Re:Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-15 Thread zst...@163.com
Hi Qingsheng and devs, Thanks for your heated discussion and redesign to optmize this feature. I just have two comments: 1. How about abtract the LookupCache to a higher level with a common Cache? It will be convenient for devs to use in other place. 2. Does it have any metrics, such as

Kinesis Sink - Data being received with intermittent breaks

2022-05-15 Thread Zain Haider Nemati
Hi, Im fetching data from kafka topics converting them to chunks of <= 1MB and sinking them to a kinesis data stream. The streaming job is functional however I see bursts of data in kinesis stream with intermittent dips where data received is 0. I'm attaching the configuration parameters for kinesi

[jira] [Created] (FLINK-27617) Add more analysis functions for Flink batch

2022-05-15 Thread luoyuxia (Jira)
luoyuxia created FLINK-27617: Summary: Add more analysis functions for Flink batch Key: FLINK-27617 URL: https://issues.apache.org/jira/browse/FLINK-27617 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-27618) Suuport CumeDist

2022-05-15 Thread luoyuxia (Jira)
luoyuxia created FLINK-27618: Summary: Suuport CumeDist Key: FLINK-27618 URL: https://issues.apache.org/jira/browse/FLINK-27618 Project: Flink Issue Type: Sub-task Reporter: luoyuxia

[jira] [Created] (FLINK-27619) Support NTIE

2022-05-15 Thread luoyuxia (Jira)
luoyuxia created FLINK-27619: Summary: Support NTIE Key: FLINK-27619 URL: https://issues.apache.org/jira/browse/FLINK-27619 Project: Flink Issue Type: Sub-task Reporter: luoyuxia

[jira] [Created] (FLINK-27620) Support percent_rank

2022-05-15 Thread luoyuxia (Jira)
luoyuxia created FLINK-27620: Summary: Support percent_rank Key: FLINK-27620 URL: https://issues.apache.org/jira/browse/FLINK-27620 Project: Flink Issue Type: Sub-task Reporter: luoyu

[jira] [Created] (FLINK-27621) expose SafetyNetWrapperClassLoader from private to public and add addUrl method for flink client

2022-05-15 Thread jackylau (Jira)
jackylau created FLINK-27621: Summary: expose SafetyNetWrapperClassLoader from private to public and add addUrl method for flink client Key: FLINK-27621 URL: https://issues.apache.org/jira/browse/FLINK-27621

Re: [DISCUSS] FLIP-232: Add Retry Support For Async I/O In DataStream API

2022-05-15 Thread Jingsong Li
Thank Lincoln for the proposal. ## Motivation: > asyncInvoke and callback functions are executed synchronously by the main thread, which is not suitable adding long time blocking operations, and introducing additional thread will bring extra complexity for users According to the documentation of

[jira] [Created] (FLINK-27622) Make `AsyncDataStream.OutputMode` configurable for table module

2022-05-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-27622: --- Summary: Make `AsyncDataStream.OutputMode` configurable for table module Key: FLINK-27622 URL: https://issues.apache.org/jira/browse/FLINK-27622 Project: Flink

[jira] [Created] (FLINK-27623) Add 'table.exec.async-lookup.output-mode' to ExecutionConfigOptions

2022-05-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-27623: --- Summary: Add 'table.exec.async-lookup.output-mode' to ExecutionConfigOptions Key: FLINK-27623 URL: https://issues.apache.org/jira/browse/FLINK-27623 Project: Flink

[jira] [Created] (FLINK-27624) Add plan validation when 'table.exec.async-lookup.output-mode' is unordered

2022-05-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-27624: --- Summary: Add plan validation when 'table.exec.async-lookup.output-mode' is unordered Key: FLINK-27624 URL: https://issues.apache.org/jira/browse/FLINK-27624 Project: Fl

[jira] [Created] (FLINK-27625) Add query hint for async lookup join

2022-05-15 Thread lincoln lee (Jira)
lincoln lee created FLINK-27625: --- Summary: Add query hint for async lookup join Key: FLINK-27625 URL: https://issues.apache.org/jira/browse/FLINK-27625 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27626) Introduce pre-aggregated merge to table store

2022-05-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-27626: Summary: Introduce pre-aggregated merge to table store Key: FLINK-27626 URL: https://issues.apache.org/jira/browse/FLINK-27626 Project: Flink Issue Type: New

[jira] [Created] (FLINK-27627) Incorrect result when order by (string, double) pair with NaN values

2022-05-15 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-27627: --- Summary: Incorrect result when order by (string, double) pair with NaN values Key: FLINK-27627 URL: https://issues.apache.org/jira/browse/FLINK-27627 Project: Flink

Re: [DISCUSS] FLIP-232: Add Retry Support For Async I/O In DataStream API

2022-05-15 Thread Lincoln Lee
Hi Jinsong, Thanks for your feedback! Let me try to answer the two questions: For q1: Motivation Yes, users can implement retries themselves based on the external async client, but this requires each user to do similar things, and if we can support retries uniformly, user code would become much s

[jira] [Created] (FLINK-27628) Table Store records and fetches incorrect results with NaN

2022-05-15 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-27628: --- Summary: Table Store records and fetches incorrect results with NaN Key: FLINK-27628 URL: https://issues.apache.org/jira/browse/FLINK-27628 Project: Flink Issu

Re: [DISCUSS] FLIP-224: Blacklist Mechanism

2022-05-15 Thread Lijie Wang
Hi Konstantin, thanks for your feedback. >From what I understand, PUT should be idempotent. However, we have a *timeout* field in the request. This means that initiating the same request at two different times will lead to different resource status (timestamps of the items to be removed will be di

Re: [VOTE] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-15 Thread Jing Zhang
+1 Thanks @ Jingsong for driving this topic. Best, Jing Zhang Jingsong Li 于2022年5月12日周四 17:06写道: > Hi, everyone > > Thanks all for your attention to FLIP-226: Introduce Schema Evolution on > Table Store [1] and participation in the discussion in the mail thread [2]. > > I'd like to start a vote

Re: [VOTE] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-15 Thread Jark Wu
+1 (binding) Best, Jark On Mon, 16 May 2022 at 13:50, Jing Zhang wrote: > +1 > Thanks @ Jingsong for driving this topic. > > Best, > Jing Zhang > > Jingsong Li 于2022年5月12日周四 17:06写道: > > > Hi, everyone > > > > Thanks all for your attention to FLIP-226: Introduce Schema Evolution on > > Table S

Re: [DISCUSS] FLIP-91: Support SQL Client Gateway

2022-05-15 Thread Timo Walther
Hi Shengkai, Hi Jark, thanks for the additional explanation and the update of the FLIP. This will help us in the future for documenting our decisions. The arguments why to include the Gateway into the main repo make a lot of sense to me. Esp. also because both CLI and gateway need some parsing

[jira] [Created] (FLINK-27629) Table Store throws NullPointerException when pushing down NotEqual predicate to a column consisting of nulls

2022-05-15 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-27629: --- Summary: Table Store throws NullPointerException when pushing down NotEqual predicate to a column consisting of nulls Key: FLINK-27629 URL: https://issues.apache.org/jira/browse/FLI

Re: [DISCUSS] FLIP-232: Add Retry Support For Async I/O In DataStream API

2022-05-15 Thread Jingsong Li
Thanks Lincoln for your reply. I'm a little confused about the relationship between Ordered/Unordered Queue and DelayQueue. Why do we need to have a DelayQueue? Can we remove the DelayQueue and put the state of the retry in the StreamRecordQueueEntry (seems like it's already in the FLIP) The adva

Re: [ANNOUNCE] Problems with build due to tests using too much disk space

2022-05-15 Thread Martijn Visser
Hi all, Unfortunately, the issue has not been resolved yet. Will look into this again now (see https://issues.apache.org/jira/browse/FLINK-24433 for updates). Thanks, Martijn On Fri, 13 May 2022 at 13:52, Konstantin Knauf wrote: > Thanks a lot for taking care of this, Martijn! > > Am Fr., 13.

[jira] [Created] (FLINK-27630) maven-source-plugin for table planner values connector for debug

2022-05-15 Thread jackylau (Jira)
jackylau created FLINK-27630: Summary: maven-source-plugin for table planner values connector for debug Key: FLINK-27630 URL: https://issues.apache.org/jira/browse/FLINK-27630 Project: Flink Iss

[jira] [Created] (FLINK-27631) Datastream job combined with table job

2022-05-15 Thread waywtdcc (Jira)
waywtdcc created FLINK-27631: Summary: Datastream job combined with table job Key: FLINK-27631 URL: https://issues.apache.org/jira/browse/FLINK-27631 Project: Flink Issue Type: Bug Comp