The timeline of Flink support Reactive mode in Yarn

Hi, community. For streaming applications, auto scaling is critical to guarantee our resource provision, not big or small. i.e. it is cost efficiency requirements from public cloud (AWS) players. I read the document of FLIP-159

Re: [DISCUSS] Component labels in PR/commit messages

+1 to Till's proposal to update the wording. Regarding c) The guide [1] actually mentions a good heuristic for coming up with a label that is also suitable for newcomers: The maven module name where most of the changes are. [1] https://flink.apache.org/contributing/code-style-and-quality-pull-req

Re: [DISCUSS] Feedback Collection Jira Bot

+1 This would also cover test instabilities, which I personally believe should not be auto-deprioritized until they've been analyzed. On Wed, May 19, 2021 at 1:46 PM Till Rohrmann wrote: > I like this idea. +1 for your proposal Konstantin. > > Cheers, > Till > > On Wed, May 19, 2021 at 1:30 PM K

Re: The timeline of Flink support Reactive mode in Yarn

Hey Bing, The idea of reactive mode is that it reacts to changing resource availability, where an outside service is adding or removing machines to a Flink cluster. Reactive Mode doesn't work on an active resource manager such as YARN, because you can not tell YARN from the outside to remove resou

Re: [DISCUSS] Releasing Flink 1.13.1

Hi everyone, Let's see where we stand towards releasing Flink 1.13.1. I am aware of one additional license-related blocker that popped up: https://issues.apache.org/jira/browse/FLINK-22706. Overall: Done or close to done: * https://issues.apache.org/jira/browse/FLINK-22666: Done * https://issue

[RESULT][VOTE] Release 1.12.4, release candidate #1

Dear devs, I'm happy to announce that we have unanimously approved this release. There are 7 approving votes, 4 of which are binding: * Roman Khachatryan * Dawid Wysakowicz (binding) * Robert Metzger (binding) * Xingbo Huang * Chesnay Schepler (binding) * Leonard Xu * Thomas Weise (binding) Ther

Re: [DISCUSS] Releasing Flink 1.13.1

* https://issues.apache.org/jira/browse/FLINK-22686: This is marked as Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make this a blocker for this bug fix as it only affects unaligned checkpoints in combination with broadcast state. What do you think? Can you already give an est

[jira] [Created] (FLINK-22721) Breaking HighAvailabilityServices interface by adding new method

Till Rohrmann created FLINK-22721: - Summary: Breaking HighAvailabilityServices interface by adding new method Key: FLINK-22721 URL: https://issues.apache.org/jira/browse/FLINK-22721 Project: Flink

Re: [DISCUSS] Releasing Flink 1.13.1

Thanks Dawid for helping with the release & Thanks Konstantin for the summary! > * https://issues.apache.org/jira/browse/FLINK-22266: Assigned to Robert. No PR yet. There's actually a PR (Jira is not showing all links by default, you need to click "show more"), and it is somewhat close to comple

[jira] [Created] (FLINK-22722) Add Documentation for Kafka New Source

Qingsheng Ren created FLINK-22722: - Summary: Add Documentation for Kafka New Source Key: FLINK-22722 URL: https://issues.apache.org/jira/browse/FLINK-22722 Project: Flink Issue Type: Improvem

[jira] [Created] (FLINK-22723) multi-field-dynamic index is not supported by elasticsearch connector

Xiang Yang created FLINK-22723: -- Summary: multi-field-dynamic index is not supported by elasticsearch connector Key: FLINK-22723 URL: https://issues.apache.org/jira/browse/FLINK-22723 Project: Flink

[jira] [Created] (FLINK-22724) NoSuchMethodError PackagedProgramUtils.isPython (Application mode in native kubernetes)

HYUNHOO KWON created FLINK-22724: Summary: NoSuchMethodError PackagedProgramUtils.isPython (Application mode in native kubernetes) Key: FLINK-22724 URL: https://issues.apache.org/jira/browse/FLINK-22724

[jira] [Created] (FLINK-22725) SlotManagers should unregister metrics at the start of suspend()

Chesnay Schepler created FLINK-22725: Summary: SlotManagers should unregister metrics at the start of suspend() Key: FLINK-22725 URL: https://issues.apache.org/jira/browse/FLINK-22725 Project: Fli

[jira] [Created] (FLINK-22726) Hive GROUPING__ID returns different value in older versions

Rui Li created FLINK-22726: -- Summary: Hive GROUPING__ID returns different value in older versions Key: FLINK-22726 URL: https://issues.apache.org/jira/browse/FLINK-22726 Project: Flink Issue Type:

[jira] [Created] (FLINK-22727) SerializableAvroSchema cannot handle large schema

Serge Travin created FLINK-22727: Summary: SerializableAvroSchema cannot handle large schema Key: FLINK-22727 URL: https://issues.apache.org/jira/browse/FLINK-22727 Project: Flink Issue Type:

[jira] [Created] (FLINK-22728) a problem of loading udf

JYXL created FLINK-22728: Summary: a problem of loading udf Key: FLINK-22728 URL: https://issues.apache.org/jira/browse/FLINK-22728 Project: Flink Issue Type: Bug Components: API / Python

Issue with using siddhi extension function with flink

Hi ,    i am trying to integrate siddhi with flink while trying to use siddhi extension function on deploying the job in flink cluster it is not able to find those libraries at run time , so i had to explicitly put those libraries to the /opt/flink/lib folder for the jobmanager and taskmanager

[jira] [Created] (FLINK-22729) Truncated Messages in Python workers

Stephan Ewen created FLINK-22729: Summary: Truncated Messages in Python workers Key: FLINK-22729 URL: https://issues.apache.org/jira/browse/FLINK-22729 Project: Flink Issue Type: Bug

Re: [Statefun] Truncated Messages in Python workers

Thanks for reporting this, it looks indeed like a potential bug. I filed this Jira for it: https://issues.apache.org/jira/browse/FLINK-22729 Could you share (here ot in Jira) what the stack on the Python Worker side is (for example which HTTP server)? Do you know if the message truncation happens

[jira] [Created] (FLINK-22730) Lookup join condition with CURRENT_DATE fails to filter records

Caizhi Weng created FLINK-22730: --- Summary: Lookup join condition with CURRENT_DATE fails to filter records Key: FLINK-22730 URL: https://issues.apache.org/jira/browse/FLINK-22730 Project: Flink

Re: [DISCUSS] Component labels in PR/commit messages

Thanks for raising this issue. I agree with the above points. One simple argument against labels is that they consume space in the commit messages. +1 to make labels optional Regards, Roman On Thu, May 20, 2021 at 9:31 AM Robert Metzger wrote: > > +1 to Till's proposal to update the wording. >

[jira] [Created] (FLINK-22731) Environment variables are not passed to TM (YARN)

Roman Khachatryan created FLINK-22731: - Summary: Environment variables are not passed to TM (YARN) Key: FLINK-22731 URL: https://issues.apache.org/jira/browse/FLINK-22731 Project: Flink I

Re: [DISCUSS] Watermark propagation with Sink API

Filed FLIP-167: Watermarks for Sink API: https://cwiki.apache.org/confluence/display/FLINK/FLIP-167%3A+Watermarks+for+Sink+API I'd like to call a vote next week, is that reasonable? On Wed, May 19, 2021 at 6:28 PM Zhou, Brian wrote: > Hi Arvid and Eron, > > Thanks for the discussion and I read

[jira] [Created] (FLINK-22732) Restrict ALTER TABLE from setting empty table options

Jane Chan created FLINK-22732: - Summary: Restrict ALTER TABLE from setting empty table options Key: FLINK-22732 URL: https://issues.apache.org/jira/browse/FLINK-22732 Project: Flink Issue Type: S

[jira] [Created] (FLINK-22733) Type mismatch thrown in KeyedStream.union in Python DataStream API

Dian Fu created FLINK-22733: --- Summary: Type mismatch thrown in KeyedStream.union in Python DataStream API Key: FLINK-22733 URL: https://issues.apache.org/jira/browse/FLINK-22733 Project: Flink Iss

Re: [DISCUSS]FLIP-150: Introduce Hybrid Source

Hi Nicholas, Thanks for taking a look at the PR! 1. Regarding switching mechanism: There has been previous discussion in this thread regarding the pros and cons of how the switching can be exposed to the user. With fixed start positions, no special switching interface to transfer information be

[jira] [Created] (FLINK-22734) ExtractionUtils#getClassReader causes "open too many files" error

zhiping lin created FLINK-22734: --- Summary: ExtractionUtils#getClassReader causes "open too many files" error Key: FLINK-22734 URL: https://issues.apache.org/jira/browse/FLINK-22734 Project: Flink

[jira] [Created] (FLINK-22735) HiveTableSourceITCase.testStreamPartitionReadByCreateTime failed because of times out

Guowei Ma created FLINK-22735: - Summary: HiveTableSourceITCase.testStreamPartitionReadByCreateTime failed because of times out Key: FLINK-22735 URL: https://issues.apache.org/jira/browse/FLINK-22735