Re: [Vote] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

+1 On 18/01/2021 15:50, Seth Wiesman wrote: > Addendum, 72 hours from now is Thursday the 21st :) > > sorry for the mistake. > > Seth > > On Mon, Jan 18, 2021 at 8:41 AM Timo Walther wrote: > >> +1 >> >> Thanks for upgrading our docs infrastructure. >> >> Regards, >> Timo >> >> On 18.01.21 15:29,

Re: [DISCUSS] Removal of flink-swift-fs-hadoop module

+1 On 27/01/2021 11:00, Chesnay Schepler wrote: > +1 > > While I don't think the filesystem is really useless, in it's current > state it is not maintainable, and so far no one rose up to remedy this. > > On 1/27/2021 10:57 AM, Aljoscha Krettek wrote: >> +1 >> >> I'm almost always in favour of rem

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

Hi Timo, From my perspective the proposed changes look good. I agree it is an important step towards FLIP-129 and FLIP-136. Personally I feel comfortable voting on the document. Best, Dawid On 05/02/2021 16:09, Timo Walther wrote: > Hi everyone, > > you might have seen that we discussed a bette

Re: [ANNOUNCE] Welcome Roman Khachatryan a new Apache Flink Committer

Congratulations Roman! Glad to have you on board! Best, Dawid On 10/02/2021 14:44, Igal Shilman wrote: > Welcome Roman! > Top-notch stuff! :) > > All the best, > Igal. > > On Wed, Feb 10, 2021 at 2:15 PM Kostas Kloudas wrote: > >> Congrats Roman! >> >> Kostas >> >> On Wed, Feb 10, 2021 at 2:08

Re: [VOTE] FLIP-164: Improve Schema Handling in Catalogs

+1 (binding) Best, Dawid On 12/02/2021 13:33, Timo Walther wrote: > Hi everyone, > > I'd like to start a vote on FLIP-164 [1] which was discussed in [2]. > > The vote will be open for at least 72 hours. Unless there are any > objections, I'll close it by February 17th, 2021 (due to weekend) if >

[UPDATE] Release 1.13 feature freeze

Hi all, The agreed date of a feature freeze is due in about a month. Therefore we thought it would be a good time to give an update of the current progress. From the information we gathered there are currently no known obstacles or foreseeable delays. We are still aiming for the end of March as t

Re: [DISCUSS] Splitting User support mailing list

As others I'd also rather be -1 on splitting (even splitting out the statefun). Personally I don't find it problematic. I often find the subjects quite descriptive, they often include tags or mention which API they refer to. If they don't I am quite sure having separate sub-lists would not help in

Re: [DISCUSS] Apache Flink Jira Process

Hi Konstantin, I also like the idea. Two comments: * you describe the "Trivial" priority as one that needs to be implemented immediately. First of all it is not used to often, but I think the way it works now is similar with a "starter" label. Tasks that are not bugs, are easy to implement and w

Re: [DISCUSS] Apache Flink Jira Process

n and use the "starter" label more aggressively? > > Best, > > Konstantin > > On Mon, Mar 1, 2021 at 11:44 AM Dawid Wysakowicz > wrote: > >> Hi Konstantin, >> >> I also like the idea. >> >> Two comments: >> >> * you desc

Re: [RESULT] [VOTE] Release 1.12.2, release candidate #2

Yes, Henry is right. Binding votes on Apache releases must come from PMC members. Out of the votes 4 are binding: * Kurt Young * Piotr Nowojski * Yu Li * Zhu Zhu Best, Dawid On 02/03/2021 21:17, Henry Saputra wrote: > Roman, binding Votes only come from Apache Flink PMC members. > > From the li

Re: [ANNOUCE][DISCUSS] Roadmap Update for Website

Thank you Stephan for working on this. It's really nice to see the roadmap updated. I also find the "Feature Radar" extremely helpful. It looks good to be published from my side. Best, Dawid On 02/03/2021 15:38, Stephan Ewen wrote: > Hi all! > > The roadmap on the Flink website is quite outdated

[DISCUSS] Feature freeze date for 1.13

Hi devs, users! 1. *Feature freeze date* We are approaching the end of March which we agreed would be the time for a Feature Freeze. From the knowledge I've gather so far it still seems to be a viable plan. I think it is a good time to agree on a particular date, when it should happen. We suggest

Re: [DISCUSS] Feature freeze date for 1.13

le changes and for backports. The following should > help to safeguard from the issue encountered previously: > https://github.com/jazzband/pip-tools/issues/1085 > > Thanks, > Thomas > > > On Tue, Mar 23, 2021 at 4:58 AM Dawid Wysakowicz > wrote: > >> Hi devs, u

Re: [VOTE] Release flink-shaded 13.0, release candidate #0

+1 - Checked diffs between 1.12 and 1.11: https://github.com/apache/flink-shaded/compare/release-12.0...release-13.0-rc1 - reviewed the website PR - verified release notes and that the release contain the changes -

Re: [DISCUSS] Feature freeze date for 1.13

; > > > > > > Regarding the "Rebase and merge" button. I find that > merge option > > useful, > > > > especially for small simple changes and for > backports. The following >

[ANNOUNCE] Release 1.13.0, release candidate #0

itory [3],| |* source code tag "release-1.2.3-rc3" [4],|   |Your help testing the release will be greatly appreciated! |   |Thanks,| |Dawid Wysakowicz |   |[1] https://dist.apache.org/repos/dist/dev/flink/flink-1.13.0-rc0/ | |[2] https://dist.apache.org/repos/dist/release/fl

[ANNOUNCE] Release 1.13 branch

Hi committers! I'd like to remind that we agreed we won't cut off the 1.13 branch until we are a bit more confident with the stability of the master branch. Therefore I'd kindly ask not to push the release-1.13 branch yet. I did push that branch accidentally when creating the rc0, but I reported

Re: [DISCUSS] Merge StreamTableEnvironment.to/fromChangelogStream to release-1.13

Hi Timo, First of all, thanks for giving a good example of following the community agreements ;) Do you have an ETA for when the feature can be merged? Given that as you are saying, it mostly about adding a new API that would be marked as experimental and is rather well isolated from the rest of

Re: [VOTE] Release 1.13.0, release candidate #1

Hi, Since publishing the RC1 we've fixed a number of issues including few critical ones such as e.g. FLINK-22001, FLINK-22354, FLINK-22369 to name a few. Therefore I'm cancelling this vote and I will prepare a new RC shortly. Best, Dawid On 18/04/2021 22:54, Guowei Ma wrote: > Hi everyone, > >

[VOTE] Release 1.13.0, release candidate #2

lease-1.13.0-rc2" [5],| |* website pull request listing the new release and adding announcement blog post [6]. |   |The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes.|   |Thanks,| |Dawid Wysakowicz |   |[1] https://issues.apach

[DISCUSS] Using timeouts in JUnit tests

Hi devs! I wanted to bring up something that was discussed in a few independent groups of people in the past days. I'd like to revise using timeouts in our JUnit tests. The suggestion would be not to use them anymore. The problem with timeouts is that we have no thread dump and stack traces of the

Re: [VOTE] Release 1.12.3, release candidate #1

+1 (binding) - Verified checksums and signatures - Reviewed the website PR - Built from sources - verified dependency version upgrades and updates in NOTICE files compared to 1.12.2 - started cluster and run WordCount example in BATCH mode and everything looked good On 23/04/2021 23:52, Arvid H

Re: [VOTE] Release 1.13.0, release candidate #2

Hey Matthias, I'd like to double confirm what Guowei said. The dependency is Apache 2 licensed and we do not bundle it in our jar (as it is in the runtime scope) thus we do not need to mention it in the NOTICE file (btw, the best way to check what is bundled is to check the output of maven shade p

Re: [VOTE] Release 1.13.0, release candidate #2

>>> 2021年4月29日 下午3:11，Jark Wu 写道： >>>>> >>>>> +1 (binding) >>>>> >>>>> - checked/verified signatures and hashes >>>>> - started cluster and run some e2e sql queries using SQL Client, >> results >>>>&

[RESULT][VOTE] Release 1.13.0, release candidate #2

I am happy to announce that we have approved the 1.13.0 release. Thank you all for making it happen! There were 12 approving votes, 5 of which were binding * Piotr Nowojski (binding) * Zhu Zhu (binding) * Jark Wu (binding) * Dian Fu (binding) * Robert Metzger (binding) * Xintong Song

[ANNOUNCE] Apache Flink 1.13.0 released

|The Apache Flink community is very happy to announce the release of Apache Flink 1.13.0.|   |Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications.|   |The release is available for download at:| |htt

Re: [DISCUSS] Using timeouts in JUnit tests

t;>> On Tue, Apr 27, 2021 at 3:49 PM Dong Lin >>>> wrote: >>>>>>>> There is one more point that may be useful to consider here. >>>>>>>> >>>>>>>> In order to debug deadlock that is not easily reproducible, i

Re: About the windowOperator and Watermark

Hi, When a Watermark arrives the window operator will emit all windows that are considered finished at the time of the Watermark. In your example (assuming both windows are finished) they will both be emitted. Best, Dawid On 08/05/2021 08:03, 曲洋 wrote: > Hi Experts, > > Given that a window in t

Re: [VOTE] Release 1.12.4, release candidate #1

+1 (binding) * Verified checksums and signatures * Checked no significant version changes compared to 1.12.3 (one new test scope dependency) * Checked no changes to the NOTICE files * Built from sources * Run example using binary 2.12 distribution * verified a random class in flink

Re: [DISCUSS] Releasing Flink 1.13.1

* https://issues.apache.org/jira/browse/FLINK-22686: This is marked as Blocker, but there is no one assigned yet. Piotr/Arvid: I wouldn't make this a blocker for this bug fix as it only affects unaligned checkpoints in combination with broadcast state. What do you think? Can you already give an est

Re: [DISCUSS] Releasing Flink 1.13.1

the release & Thanks Konstantin for the > summary! > > > > * https://issues.apache.org/jira/browse/FLINK-22266 > <https://issues.apache.org/jira/browse/FLINK-22266>: Assigned to > Robert. > No PR yet. > > There's actually a PR (Ji

[VOTE] Release 1.13.1, release candidate #1

|Hi everyone,| |Please review and vote on the release candidate #1 for the version 1.13.1, as follows:| |[ ] +1, Approve the release| |[ ] -1, Do not approve the release (please provide specific comments)|     |The complete staging area is available for your review, which includes:| |* JIRA release

Re: [VOTE] Release 1.13.1, release candidate #1

s: >> >> >> >> >> - downloaded and build source code >> - started cluster and succeed in jobs (flink applications defined by me) >> - checked diff of pom files between 1.13.1-rc1 and 1.13.0 >> >> >> >> >> >> >> >> Best, Ro

Re: [VOTE] Release 1.13.1, release candidate #1

id a visual check of the release blog post > - started cluster and ran jobs (WindowJoin and WordCount); nothing > suspicious found in the logs > - verified change FLINK-22866 manually whether the issue is fixed > > Best, > Matthias > > On Tue, May 25, 2021 at 3:33 PM Dawid Wysakow

Re: [VOTE] Release 1.13.1, release candidate #1

> - built from source code > > - check apache-flink source/wheel package content > > - run python udf job > > > > Best, > > Xingbo > > > > Dawid Wysakowicz <mailto:dwysakow...@apache.org> <mailto:dwysakow...@a

[RESULT][VOTE] Release 1.13.1, release candidate #1

Dear devs, I'm happy to announce that we have unanimously approved this release. There are 8 approving votes, 3 of which are binding: *     Matthias Pohl *     Roc Marshal *     Jing Zhang *     Chesnay Schepler (binding) *     Dawid Wysakowicz (binding) *     Xingbo

[ANNOUNCE] Apache Flink 1.13.1 released

/05/28/release-1.13.1.html|   |The full release notes are available in Jira:| |https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12350058|   |We would like to thank all contributors of the Apache Flink community who made this release possible!|   |Regards,| |Dawid

Re: [ANNOUNCE] Apache Flink 1.13.1 released

.docker.com/_/flink > > Hope it'll be available soon. > > Thanks, > Youngwoo > > > On Sat, May 29, 2021 at 1:49 AM Dawid Wysakowicz > wrote: > >> The Apache Flink community is very happy to announce the release of Apache >> Flink 1.13.1, which is the firs

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

Hi all, Thanks for the very insightful discussion. I'd like to revive the effort of FLIP-147. First of all, from my side I'd like to say that I am really interested in helping that happen in the upcoming 1.14 release. I agree with Till that the final checkpoints and global commits are mostly orth

Re: [VOTE] Watermark propagation with Sink API

+1 (binding) Best, Dawid On 03/06/2021 03:50, Zhou, Brian wrote: > +1 (non-binding) > > Thanks Eron, looking forward to seeing this feature soon. > > Thanks, > Brian > > -Original Message- > From: Arvid Heise > Sent: Wednesday, June 2, 2021 15:44 > To: dev > Subject: Re: [VOTE] Waterma

Re: [DISCUSS] Watermark propagation with Sink API

Hi Eron, I might be missing some background on Pulsar partitioning but something seems off to me. What is the chunk/batch/partition that Pulsar brokers will additionally combine watermarks for? Isn't it the case that only a single Flink sub-task would write to such a chunk and thus will produce an

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

Hey, @Arvid The problem with adding the "drain/flush/stopProcessing" method to RichFunction is that it is not typed with the output type. At the same time we would most likely need a way to emit records from the method. That's originally thought about adding a typed interface which honestly I don'

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

; impression that someone wanted to reinvent a wheel :) `flush()` is kind >> of >> >>> an industry standard for things like that. Furthermore I don't think >>> `drain()` solves Till's concern (drain != stop + flush). `stopAndFlush()` >>> would be bette

[ANNOUNCE] New PMC member: Arvid Heise

Hi all! I'm very happy to announce that Arvid Heise has joined the Flink PMC! Congratulations and welcome Arvid! Best, Dawid OpenPGP_signature Description: OpenPGP digital signature

[ANNOUNCE] New PMC member: Xintong Song

Hi all! I'm very happy to announce that Xintong Song has joined the Flink PMC! Congratulations and welcome Xintong! Best, Dawid OpenPGP_signature Description: OpenPGP digital signature

[DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

Hi devs, Quite recently we pushed, in our opinion, quite an important fix[1] for unaligned checkpoints which disables UC for broadcast partitioning. Without the fix there might be some broadcast state corruption. Therefore we think it would be beneficial to release it soonish. What do you think? D

Re: CONTENTS DELETED in nabble frontend

Hey, As far as I know the official Apache ML archive can be accessed here[1]. Personally I don't know what is the status of the nabble archives. Best, Dawid [1] https://lists.apache.org/list.html?dev@flink.apache.org On 23/06/2021 09:08, Matthias Pohl wrote: > Thanks for pointing to the Nabble

Re: [VOTE] FLIP-147: Support Checkpoint After Tasks Finished

+1 (binding) Best, Dawid On 28/06/2021 10:45, Yun Gao wrote: > Hi all, > > For FLIP-147[1] which targets at supports checkpoints after tasks finished > and modify operator > API and implementation to ensures the commit of last piece of data, since > after the last vote > we have more discussi

Re: [VOTE] Release 1.13.2, release candidate #1

+1 (binding) * verified signatures and checksums * reviewed the announcement PR * built from sources and run an example, quickly checked Web UI * checked diff of pom.xml and NOTICE files from 1.13.1, o commons-io updated, o bundled guava:failureaccess addded in flink-sq

Re: [VOTE] FLIP-172: Support custom transactional.id prefix in FlinkKafkaProducer

+1 (binding) Best, Dawid On 28/06/2021 16:00, Wenhao Ji wrote: > Hi everyone, > > I would like to start a vote on FLIP-172 [1] which was discussed in > this thread [2]. > The vote will be open for at least 72 hours until July 1 unless there > is an objection or not enough votes. > > Thanks, > We

Re: [RESULT][VOTE] FLIP-147: Support Checkpoint After Tasks Finished

t;>> snapshotState() method so that we could shut down the whole topology >>> with >>>>> a single final checkpoint. >>>>> Very sorry for not include enough details for this part and I'll >>>>> complement the FLIP with the detail

Re: [RESULT][VOTE] FLIP-147: Support Checkpoint After Tasks Finished

eckpoint? Couldn't we do the computation in a different thread in order > >> to relieve the main thread a bit. > >> > >> 2) The implementation of the DefaultCheckpointPlanCalculator seems to go > >> over the whole topology for every calculation. Wouldn&#

Re: [RESULT][VOTE] FLIP-147: Support Checkpoint After Tasks Finished

> |||}| > |}| > | > | > > I think we should still keep the change ?  > > Best, > Yun > > --Original Mail -- > *Sender:*Dawid Wysakowicz > *Send Date:*Sun Jul 18 18:44:50 2021 > *Recipients:*Flink Dev , Yun Gao >

Re: [RESULT][VOTE] FLIP-147: Support Checkpoint After Tasks Finished

gt; records coming only control events could make sense. > > I would leave the proposed optimization out of the first version. We can > still add it at a later point in time. > > Cheers, > Till > > On Mon, Jul 19, 2021 at 10:35 AM Dawid Wysakowicz > wrote: > >>

Re: [RESULT][VOTE] FLIP-147: Support Checkpoint After Tasks Finished

Small correction. I meant we need to adjust the EndOfInputEvent of course. Best, Dawid On 19/07/2021 11:48, Dawid Wysakowicz wrote: > Hey Till, > > Yes, you're right we will have to adjust the current state of > EndOfPartitionEvent and move the moment when we emit it to

Re: [VOTE] Release flink-shaded 14.0, release candidate 1

+1 (binding) * checked the website PR * verified changed versions and NOTICE files since version 13.0 * verified the checksum and the signature Best, Dawid On 19/07/2021 10:58, Timo Walther wrote: > +1 (binding) > > I went through all commits one more time and could not spot anything > th

Re: [RESULT][VOTE] FLIP-147: Support Checkpoint After Tasks Finished

> Tasks Finished > > Small correction. I meant we need to adjust the EndOfInputEvent of course. > > Best, > > Dawid > > On 19/07/2021 11:48, Dawid Wysakowicz wrote: > > Hey Till, > > > > Yes, you're right we will have 

Re: [RESULT][VOTE] FLIP-147: Support Checkpoint After Tasks Finished

; impossible to implement a real life use case. Theoretically yes, the user > > can store `endInput()` on state, and after rescaling this state would be > > inconsistent with what is actually happening with the operator, but I > > haven't found a use case that would bre

Re: [DISCUSS] FLIP-180: Adjust StreamStatus and Idleness definition

t;>> going >>>>> idle when temporary no splits are assigned. I will include it in the >>>> FLIP. >>>>> I agree with adding the markActive() functionality, for symmetry. >>>> Speaking >>>>>> of symmetry, could we now includ

Flink 1.14 Weekly 2021-09-07

Dear Apache Flink community, Today was another update and we are making a good progress towards finally having the release. *Blockers, critical issues, cross-team testing and documentation* All of the release blockers are assigned and are making a good progress. Similarly the cross-team testing i

[DISCUSS] Purpose of release notes

Hi devs, When working on the release notes for the upcoming 1.14.0[1] release I spotted inconsistencies in where we add release-notes. Therefore I wanted to ask how do we see the purpose of this page. So far the first paragraph states: These release notes discuss important aspects, such as

[VOTE] Release 1.14.0, release candidate #1

|Hi everyone,| |Please review and vote on the release candidate #1 for the version 1.14.0, as follows:| |[ ] +1, Approve the release| |[ ] -1, Do not approve the release (please provide specific comments)|     |The complete staging area is available for your review, which includes:| |* JIRA release

Re: [VOTE] Release 1.14.0, release candidate #1

>> Sender:Kurt Young >> Date:2021/09/16 12:24:38 >> Recipient:dev >> Theme:Re: [VOTE] Release 1.14.0, release candidate #1 >> >> I noticed that a serious performance degradation has been reported [1], >> shall we wait >> for the conclusion for that iss

Re: [VOTE] Release 1.14.0, release candidate #2

Hey all, I would like to cancel the vote because of two issues we found: https://issues.apache.org/jira/browse/FLINK-24303 https://issues.apache.org/jira/browse/FLINK-24347 Especially the second one results in a not the very best out of the box experience for users. Even though it has been ther

[VOTE] Release 1.14.0, release candidate #3

Hi everyone, Please review and vote on the release candidate #3 for the version 1.14.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], *

Re: [VOTE] Release 1.14.0, release candidate #3

on3!=1.9.2,<1.10.0,>=1.8.1 > Using cached avro_python3-1.9.2.1-py3-none-any.whl > ERROR: Could not find a version that satisfies the requirement > apache-flink-libraries<1.14.1,>=1.14.0 (from apache-flink) > ERROR: No matching distribution found for > apache-flink-libr

Re: [VOTE] Release 1.14.0, release candidate #3

>>>>> also >>>>>>>> refer to [1] for more details. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Dian >>>>>>>> >>>>>>>> [1] >>>>>>>> >

[RESULT] [VOTE] Release 1.14.0, release candidate #3

I'm happy to announce that we have unanimously approved this release.   There are 7 approving votes, 3 of which are binding: * Xintong Song (binding) * Zhu Zhu (binding) * Piotr Nowojski (binding) * Yangze Guo * Jing Zhang * Matthias Pohl * Leonard Xu There are no disapproving vote

[ANNOUNCE] Apache Flink 1.14.0 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.14.0.   Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications.   The release is available for download at: https://fl

Re: [DISCUSS] Creating an external connector repository

Hey all, I don't have much to add to the general discussion. Just a single comment on: that we could adjust the bylaws for the connectors such that we need fewer PMCs to approve a release. Would it be enough to have one PMC vote per connector release? I think it's not an option. This

Reserve FLIP numbers

Hey devs, Just a quick reminder to increase the counter[1] when picking up a number for your FLIPs. Thanks ;) Best, Dawid [1] https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals#FlinkImprovementProposals-FLIPround-up OpenPGP_signature Description: OpenPGP digital

Re: [ANNOUNCE] New Apache Flink Committer - Fabian Paul

Congratulations Fabian! On 15/11/2021 15:22, Marios Trivyzas wrote: > Congrats Fabian! > > On Mon, Nov 15, 2021 at 3:02 PM Francesco Guardiani > wrote: > >> Congratulations Fabian! >> >> On Mon, Nov 15, 2021 at 2:29 PM Yun Gao >> wrote: >> >>> Congratulations Fabian! >>> >>> Best, >>> Yun >>> -

Re: [ANNOUNCE] New Apache Flink Committer - Leonard Xu

Congrats! On 12/11/2021 05:12, Jark Wu wrote: > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Leonard Xu as a new Flink > committer. > > Leonard has been a very active contributor for more than two year, authored > 150+ PRs and reviewed many PRs which is quite outstanding. > Le

Re: [ANNOUNCE] New Apache Flink Committer - Yangze Guo

Congrats! On 15/11/2021 15:22, Marios Trivyzas wrote: > Congrats Yangze! > > On Mon, Nov 15, 2021 at 1:34 PM Martijn Visser > wrote: > >> Congrats Yangze! >> >> On Mon, 15 Nov 2021 at 09:13, Yangze Guo wrote: >> >>> Thank you all very much! >>> It's my honor to work with you in such a great comm

Re: [ANNOUNCE] New Apache Flink Committer - Jing Zhang

Congrats! On 15/11/2021 14:39, Timo Walther wrote: > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Jing Zhang as a new > Flink committer. > > Jing has been very active in the Flink community esp. in the Table/SQL > area for quite some time: 81 PRs [1] in total and is also activ

[DISCUSS] FLIP-193: Snapshots ownership

Hi devs, I'd like to bring up for a discussion a proposal to clean up ownership of snapshots, both checkpoints and savepoints. The goal here is to make it clear who is responsible for deleting checkpoints/savepoints files and when can that be done in a safe manner. Looking forward for your feedb

Re: [VOTE] Release 1.15.2, release candidate #2

+1 (binding) - signatures & checksums OK - checked changed licenses from 1.15.1 - PR OK - no excessive or binary files in the source distribution Best, Dawid On 19.08.2022 10:30, Xingbo Huang wrote: +1 (non-binding) - verify signatures and checksums - no binaries found in source archive - re

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

Hey Lincoln, Thanks for opening the discussion. To be honest I am not convinced if emitting from close there is a contract that was envisioned and thus should be maintained. As far as I can see it does affect only the TableFunction, because it has the collect method. None of the other UDFs (S

Re: [VOTE] Release flink-shaded 16.0, release candidate #1

+1 (binding) - Verified that checksums and GPG files match the released files - Checked that POM files point to the same version - Verified licenses - Verified web PR Best, Dawid On 29/09/2022 09:22, Martijn Visser wrote: Thanks for pushing this release Chesnay. +1 (binding) - Verified that

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

Hi Han, I think in principle your proposal makes sense and the compatibility check indeed should be done in the opposite direction. However, I have two suggestions: 1. Should we introduce the new method to the TypeSerializerSnapshot instead? E.g.    TypeSerializerSnapshot {        TypeSer

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

. Users may also need to migrate their own external serializers which is a long run. I'd like to make jobs where users could always work without modifying any codes before removing the deprecated method so I provide a default implementation in the proposal. On Mon, Oct 17, 2022 at 10

Re: [DISCUSS] Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0

+1, I think that is a sensible think to do. I don't think this will affect many users as those versions are already quite old. As a side note, there is a different effort around serializers that might introduce another incompatibility (in the API). I wonder if we could squash it together someh

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

ncept, which also can be used for compatibility. ` TypeSerializerSnapshot { TypeSerializerSchemaCompatibility resolveSchemaCompatibility( TypeSerializerSnapshot oldSnapshot); } ` Best, Godfrey Dawid Wysakowicz 于2022年10月20日周四 16:59写道： That's the final status we will arrive at. IIUC, we

Re: [VOTE] FLIP-263: Improve resolving schema compatibility

+1, Best, Dawid On 28/10/2022 08:08, godfrey he wrote: +1 (binding) Thanks for driving this! Best, Godfrey Yun Gao 于2022年10月28日周五 13:50写道： +1 (binding) Thanks Hangxiang for driving the FLIP. Best, Yun Gao --Original Mail -- Sender:Zakelly Lan Send

Re: [VOTE] Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0

+1 On 28/10/2022 16:57, Piotr Nowojski wrote: Hi, As discussed on the dev mailing list [0] I would like to start a vote to drop support of older savepoint formats (for Flink versions older than 1.8). You can find the original explanation from the aforementioned dev mailing list thread at the bo

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

Hey all, Sorry for being rather late to the party. I'd like to chime in a few thoughts. I agree there are at least two aspects to the problem: 1) lower level handling (StreamOperator) 2) higher level APIs (WindowOperator, CEP, ProcessFunction, ...) First is how do we want to deal with it on

Re: Stack Overflow Question re compressing broadcast state

g. Hoping someone (like Dawid Wysakowicz) can chime in here, thanks! — Ken -- Ken Krugler http://www.scaleunlimited.com Custom big data solutions Flink, Pinot, Solr, Elasticsearch OpenPGP_0x31D2DD10BFC15A2D.asc Description: OpenPGP public key OpenPGP_signature

Re: Stack Overflow Question re compressing broadcast state

hanks! — Ken On Nov 17, 2022, at 12:06 AM, Dawid Wysakowicz wrote: Cross posting answer from SO: BroadcastState is an operator state not a KeyedState. The referenced docs refer to a KeyedState: Compression works on the granularity of key-groups in keyed state, Probably docs could be more exp

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

I see the problem differently. I really don't think this should be looked at from the perspective of timers, but rather from the semantics of endOfInput. If a proc time timer is set for 10s, the 10s are "real-time" and the delay is important. One particularly important example is "wait

Re: Stack Overflow Question re compressing broadcast state

. On Nov 21, 2022, at 1:36 AM, Dawid Wysakowicz wrote: And yes, I read "Compression works on the granularity of key-groups in keyed state” as meaning “When compressing keyed state, it’s done per key-group” and not “Compression only works on keyed state” :) Totally agree. Docs

Re: [DISCUSS] Hierarchies in ConfigOption

Hi all, I also wanted to share my opinion. When talking about a ConfigOption hierarchy we use for configuring Flink cluster I would be a strong advocate for keeping a yaml/hocon/json/... compatible style. Those options are primarily read from a file and thus should at least try to follow common p

Re: [DISCUSS] flink-connector-rabbitmq api changes

Hi Karim, Sorry you did not have the best first time experience. You certainly did everything right which I definitely appreciate. The problem in that particular case, as I see it, is that RabbitMQ is not very actively maintained and therefore it is not easy too find a committer willing to take o

Re: [DISCUSS] Hierarchies in ConfigOption

; json/yaml. >> >> From the user's perspective, I don't use table configs from a config file >> like yaml or json for now, >> And it's ok to change it to yaml like style. Actually we didn't know that >> this could be a yaml like >> configurati

Re: [DISCUSS] Send issue and pull request notifications for flink-web and flink-shaded to iss...@flink.apache.org

+1 Yes, please. I've also observed a lot of noise in the past days. Best, Dawid On 04/05/2020 17:48, Tzu-Li (Gordon) Tai wrote: > +1 > > All the recent new repos, flink-statefun / flink-statefun-docker / > flink-training etc. are also sending notifications to issues@. > > Gordon > > > On Mon, M

Re: [VOTE] Release 1.10.1, release candidate #2

Hi all, I wonder if we could also include FLINK-17313 which I backported into 1.10 branch yesterday. Best, Dawid On 06/05/2020 07:26, Yu Li wrote: > Thanks Till and Thomas, will include fix for both FLINK-17496 and > FLINK-17514 in the next RC. > > Best Regards, > Yu > > > On Tue, 5 May 2020 at

Re: [DISCUSS] FLIP-126: Unify (and separate) Watermark Assigners

Hi Aljoscha, Sorry for adding comments during the vote, but I have some really minor suggestions that should not influence the voting thread imo. 1) Does it make sense to have the TimestampAssigner extend from Flink's Function? This implies it has to be serializable which with the factory pattern

Re: [DISCUSS] FLIP-126: Unify (and separate) Watermark Assigners

Best, >> Aljoscha >> >> >> [1] https://github.com/aljoscha/flink/tree/flink-xxx-wm-generators-rebased >> [2] >> >> https://github.com/aljoscha/flink/blob/flink-xxx-wm-generators-rebased/flink-core/src/main/java/org/apache/flink/api/common/eventtime/Waterma

Re: [DISCUSS] FLIP-126: Unify (and separate) Watermark Assigners

igner((event, timestamp) -> event) >>>> .build(); >>>> >>>> 3) This makes sense if we only allow WatermarkStrategies on sources, >>>> where the previous timestamp really is the "native" timestamp. >>>> Currently, we

Re: [VOTE] FLIP-126: FLIP-126: Unify (and separate) Watermark Assigners

+1 from me Best, Dawid On 12/05/2020 15:49, Stephan Ewen wrote: > +1 from me > > There are discussions about minor changes (names and serializability) > pending, but these should not conflict with the design here. > > On Tue, May 12, 2020 at 10:01 AM Aljoscha Krettek > wrote: > >> Hi all, >> >>

Re: [DISCUSS] Stability guarantees for @PublicEvolving classes

I also like the proposal for keeping the binary compatibility of @PublicEvolving for bugfix releases. As for the @Public classes I think the current guarantees are good enough. Best, Dawid On 14/05/2020 10:49, Jingsong Li wrote: > Thanks Till for starting this discussion. > > +1 for enabling th

<    1   2   3   4   5   6   7   8   9   10   >