Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-02 Thread 李宇彬
Hi Sergey It is a very useful improvement I'm looking forward to. in addition, I think prompt can play a greater role. To help users call commands in expected context, we can get session context (current catalog/db/time) from cli prompt like MySQL, please see details as below: https://iss

[jira] [Created] (FLINK-24742) SQL client add info about key strokes to docs

2021-11-02 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-24742: --- Summary: SQL client add info about key strokes to docs Key: FLINK-24742 URL: https://issues.apache.org/jira/browse/FLINK-24742 Project: Flink Issue Typ

[DISCUSS] FLIP-191: Extend unified Sink interface to support small file compaction

2021-11-02 Thread Fabian Paul
Hi all, More and more data lake sinks rely on columnar formats which benefit from few larger files than a lot of small files (read amplification). Our current FileSink cannot ensure a certain size when writing to an external filesystem which I call the small file compaction problem. Unfortunat

[jira] [Created] (FLINK-24741) Deprecate FileRecordFormat, use StreamFormat instead

2021-11-02 Thread Jing Ge (Jira)
Jing Ge created FLINK-24741: --- Summary: Deprecate FileRecordFormat, use StreamFormat instead Key: FLINK-24741 URL: https://issues.apache.org/jira/browse/FLINK-24741 Project: Flink Issue Type: Improv

[jira] [Created] (FLINK-24740) Update testcontainers dependency to the latest version

2021-11-02 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-24740: -- Summary: Update testcontainers dependency to the latest version Key: FLINK-24740 URL: https://issues.apache.org/jira/browse/FLINK-24740 Project: Flink Is

[jira] [Created] (FLINK-24739) State requirements for Flink's application mode in the documentation

2021-11-02 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24739: - Summary: State requirements for Flink's application mode in the documentation Key: FLINK-24739 URL: https://issues.apache.org/jira/browse/FLINK-24739 Project: Flink

[jira] [Created] (FLINK-24738) Fail during announcing buffer size to released local channel

2021-11-02 Thread Anton Kalashnikov (Jira)
Anton Kalashnikov created FLINK-24738: - Summary: Fail during announcing buffer size to released local channel Key: FLINK-24738 URL: https://issues.apache.org/jira/browse/FLINK-24738 Project: Flink

Re: [DISCUSS] FLIP-187: Adaptive Batch Job Scheduler

2021-11-02 Thread David Morávek
Hi, thanks for drafting the FLIP, Lijie and Zhu Zhu. It already looks pretty solid and it will be a really great improvement to the batch scheduling. I'd second to the Till's feedback, especially when it comes to the consistent behavior between different deployment types / schedulers. What I'm bit

[jira] [Created] (FLINK-24737) Update outdated web dependencies in flink-runtime-web

2021-11-02 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-24737: -- Summary: Update outdated web dependencies in flink-runtime-web Key: FLINK-24737 URL: https://issues.apache.org/jira/browse/FLINK-24737 Project: Flink Iss

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-02 Thread Sergey Nuyanzin
Hi Timo Thank you for your questions. I will answer your questions here and update FLIP's page as well >For example, who is responsible for parsing comments? I guess the SQL >Client and not the Flink SQL parser will take care of this? Yes, you are right. SQL Client is responsible for parsing her

[jira] [Created] (FLINK-24736) Non vulenerable jar files for Apache Flink 1.14.0

2021-11-02 Thread Parag Somani (Jira)
Parag Somani created FLINK-24736: Summary: Non vulenerable jar files for Apache Flink 1.14.0 Key: FLINK-24736 URL: https://issues.apache.org/jira/browse/FLINK-24736 Project: Flink Issue Type:

[jira] [Created] (FLINK-24735) SQL client crashes with `Cannot add expression of different type to set`

2021-11-02 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-24735: -- Summary: SQL client crashes with `Cannot add expression of different type to set` Key: FLINK-24735 URL: https://issues.apache.org/jira/browse/FLINK-24735 Project:

[jira] [Created] (FLINK-24734) testSecretOption fails due to missing required options

2021-11-02 Thread Anton Kalashnikov (Jira)
Anton Kalashnikov created FLINK-24734: - Summary: testSecretOption fails due to missing required options Key: FLINK-24734 URL: https://issues.apache.org/jira/browse/FLINK-24734 Project: Flink

[jira] [Created] (FLINK-24733) Data loss in pulsar source when using shared mode

2021-11-02 Thread wang shengzhi (Jira)
wang shengzhi created FLINK-24733: - Summary: Data loss in pulsar source when using shared mode Key: FLINK-24733 URL: https://issues.apache.org/jira/browse/FLINK-24733 Project: Flink Issue Typ

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-02 Thread Timo Walther
Hi Sergey, thanks for this nice demo video. It looks very nice and makes the SQL Client an even more useful tool. What I miss a bit in the FLIP is the implementation details. For example, who is responsible for parsing comments? I guess the SQL Client and not the Flink SQL parser will take c

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-02 Thread Sergey Nuyanzin
Thank you very much for the feedback. Thanks for the idea, Jark. Yes, that is possible, I added this property to the FLIP's page + same for line numbers Till, thanks for process clarification. I'm new to the FLIP process, so your comment is really helpful. I think the only unresolved thing is >W

Reactive mode in 1.13

2021-11-02 Thread Ravi Sankar Reddy Sangana
Hi team, We are planning to move to reactive mode with standalone job for scaling options. While submitting the jobs getting errors saying the kafka conusmers and client related jars are missing even they packed in the fat jar and the same jar is running with normal session cluster. Can any

Re: [DISCUSS] Over Aggregation add supports CURRENT_TIMESTAMP as the upper boundary in RANGE intervals .

2021-11-02 Thread 林挺滨
Microsoft SQL Server mainly processes offline data, so it is normal that there is no CURRENT_TIMESTAMP. Just like watermark is also a concept that exists only in real-time processing engines. I think CURRENT_TIMESTAMP is essentially the same as watermark, and both are used to trigger the window. T

[jira] [Created] (FLINK-24732) Remove scala suffix from respective benchmarks dependencies

2021-11-02 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-24732: Summary: Remove scala suffix from respective benchmarks dependencies Key: FLINK-24732 URL: https://issues.apache.org/jira/browse/FLINK-24732 Project: Flink

[jira] [Created] (FLINK-24731) Add a blank space for TransitiveClosureNaive

2021-11-02 Thread liyunchao (Jira)
liyunchao created FLINK-24731: - Summary: Add a blank space for TransitiveClosureNaive Key: FLINK-24731 URL: https://issues.apache.org/jira/browse/FLINK-24731 Project: Flink Issue Type: Improvemen

[jira] [Created] (FLINK-24730) Support prompt in SqlCli

2021-11-02 Thread Yubin Li (Jira)
Yubin Li created FLINK-24730: Summary: Support prompt in SqlCli Key: FLINK-24730 URL: https://issues.apache.org/jira/browse/FLINK-24730 Project: Flink Issue Type: Improvement Components

[jira] [Created] (FLINK-24729) Support IterationBody with mixed per-round and all-round operators

2021-11-02 Thread Yun Gao (Jira)
Yun Gao created FLINK-24729: --- Summary: Support IterationBody with mixed per-round and all-round operators Key: FLINK-24729 URL: https://issues.apache.org/jira/browse/FLINK-24729 Project: Flink Iss

[jira] [Created] (FLINK-24728) Batch SQL file sink forgets to close the output stream

2021-11-02 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-24728: --- Summary: Batch SQL file sink forgets to close the output stream Key: FLINK-24728 URL: https://issues.apache.org/jira/browse/FLINK-24728 Project: Flink Issue Ty

Re: [Discuss] Planning Flink 1.15

2021-11-02 Thread David Morávek
The contributor availability argument makes perfect sense, +1 for moving the feature freeze to 6/2. Best, D. On Thu, Oct 28, 2021 at 9:39 AM Till Rohrmann wrote: > I think it is important that most of the people who contribute new features > are available during the testing/stabilization period

[jira] [Created] (FLINK-24727) Running Kerberized YARN per-job on Docker test (no fat-jar) fails on Azure

2021-11-02 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-24727: - Summary: Running Kerberized YARN per-job on Docker test (no fat-jar) fails on Azure Key: FLINK-24727 URL: https://issues.apache.org/jira/browse/FLINK-24727 Project:

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-02 Thread Till Rohrmann
Hi Sergey, I think that after answering/resolving Jark's comments, you can start a [VOTE] thread for this FLIP. The process is described here [1]. Once the FLIP has been accepted, which it should, given the positive feedback, you can start working on it by creating the corresponding JIRA tickets a

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-02 Thread Francesco Guardiani
Hi Sergey, Thank you for your FLIP, the autocompletion looks stunning and really useful! FG On Tue, Nov 2, 2021 at 4:25 AM Jark Wu wrote: > Awesome demo, looking forward to these features! > > I only have a minor comment: could we provide a config to enable/disable > the prompt values? > We can