Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release

I've spent time exploring this playground and its accompanying documentation, and found it to be a big step forward in making it easy for folks to experience some of Flink's key features firsthand. From a training and educational perspective, I'd love to see this in 1.9.

Re: [VOTE] Apache Flink 1.9.0, release candidate #3

CoProcessFunction to a KeyedCoProcessFunction (which it should have been). I also tried a few complex queries in the SQL console, and wrote a simple job using the State Processor API. Everything worked. David David Anderson | Training Coordinator Follow us @VervericaData -- Join Flink Forward - The Apache

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

In general I'm in favor of anything that is going to make the Table API easier to learn and more predictable in its behavior. This proposal kind of falls in the middle. As someone who has spent hours in the crevices between the various flavors of the current implementations, I certainly view keepin

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

ot;, what do you think? > > Some people mentioned to overload certain methods for accepting values > or column names. E.g. `$("field").isEqual("str")` but then string values > could be confused with column names. > > Thanks, > Timo > > On 27.08.19 17:34,

Re: [NOTICE] Flink master branch now uses Maven 3.8.6

Chesnay, thank you for all your hard work on this! David On Fri, May 12, 2023 at 4:03 PM Chesnay Schepler wrote: > > > What happened? > > I have just merged the last commits to properly support Maven 3.3+ on > the Flink master branch. > > mvnw and CI have been updated to use Maven 3.8.6. >

Re: [DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

I'm delighted to see interest in developing support for processing-time temporal joins. The proposed implementation seems rather complex, and I'm not convinced this complexity is justified/necessary. I'd like to outline a simpler alternative that I think would satisfy the key objectives. Key idea

Re: [DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

s JoinOpertor to always buffer > records when isBacklog=true, whereas isBacklog's semantics only says it is > "optional" to buffer records, which can be an issue in the long term. > > Thanks, > Dong > > On Tue, Jul 25, 2023 at 2:37 AM David Anderson wro

Re: log4j2 integration with flink 1.10

Flink switched to using log4j2 with the 1.11 release. Looking at the ticket involved [1] should give you some idea of the effort involved in backporting that to 1.10. My initial impression is that you might not have to do much to get it working, The comments in [2] appear to outline what's needed.

Re: Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

+1 Keeping the legacy implementation in place is confusing and encourages adoption of something that really shouldn't be used. Thanks for driving this, David On Fri, Sep 1, 2023 at 8:45 AM Jing Ge wrote: > > Hi Wencong, > > Thanks for your clarification! +1 > > Best regards, > Jing > > On Fri,

Re: Pointers to computational models of Flink CEP

The implementation of Flink CEP was largely based on Efficient Pattern Matching over Event Streams by Jagrati Agrawal, Yanlei Diao, Daniel Gyllstrom, and Neil Immerman from UMass Amherst [1]. [1] https://people.cs.umass.edu/~yanlei/publications/sase-sigmod08.pdf Cheers, David On Tue, Oct 31, 202

Re: [DISCUSS] Change the default restart-strategy to exponential-delay

Rui, I don't have any direct experience with this topic, but given the motivation you shared, the proposal makes sense to me. Given that the new default feels more complex than the current behavior, if we decide to do this I think it will be important to include the rationale you've shared in the

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

The current situation (where we have both the legacy windows and the TVF-based windows) is confusing for users, and I'd like to see us move forward as rapidly as possible. Since the early fire, late fire, and allowed lateness features were never documented or exposed to users, I don't feel that we

Re: [DISCUSS] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

I'm delighted to see the progress on this. This is going to be a major enabler for some important use cases. The proposed simplifications (global config and ordered mode) for V1 make a lot of sense to me. +1 David On Wed, Dec 20, 2023 at 12:31 PM Alan Sheinberg wrote: > Thanks for that feedbac

Re: [ANNOUNCE] New Apache Flink Committer - Alexander Fedulov

That's great news. Congratulations, Alex! David On Tue, Jan 2, 2024 at 9:00 AM Ryan Skraba wrote: > Awesome news for the community -- congratulations Alex (and Happy New > Year everyone!) > > Ryan > > On Tue, Jan 2, 2024 at 2:55 PM Yun Tang wrote: > > > > Congratulation to Alex and Happy New Y

Re: [DISCUSS] FLIP-413: Enable unaligned checkpoints by default

Piotr, I think the situation is more nuanced than what you've described. One concern I have is that unaligned checkpoints are somewhat less flexible in terms of which operational tasks can be safely performed with them -- i.e., if you look at the table in the docs [1], aligned checkpoints support

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

We stopped using these windows scripts in our training sessions a couple of years ago, and I can't remember seeing anyone use them at all since then. While our community includes many developers using Windows for development, in my experience pretty much everyone moves to some sort of linux-based e

Re: [DISCUSS] Improvements on FLIP Process

I'd like to bring up another problem that I see with the current FLIP process, which relates to what happens after a FLIP has been implemented. A few months ago I read through all of the then existing FLIPs. My reason for doing so is that on a great many topics, the only "documentation" we have is

Re: [VOTE] [FLIP-76] Unaligned checkpoints

+1 I like where this is headed. One question: during restore, it could happen that a new task manager is configured with fewer or smaller buffers than was previously the case. How will this be handled? David On Wed, Mar 11, 2020 at 8:31 AM Arvid Heise wrote: > Hi Thomas, > > it's like you sai

Re: [DISCUSS] Drop Bucketing Sink

The BucketingSink is still somewhat widely used, I think in part because of shortcomings in the StreamingFileSink. I would hope that in tandem with removing the bucketing sink we could also address some of these issues. I'm thinking in particular of issues that are waiting on FLIP-46 [1]. Removin

[PROPOSAL] Contribute training materials to Apache Flink

Dear Flink Community! For some time now Ververica has been hosting some freely available Apache Flink training materials at https://training.ververica.com. This includes tutorial content covering the core concepts of the DataStream API, and hands-on exercises that accompany those explanations. We

Re: [PROPOSAL] Contribute training materials to Apache Flink

e that flink.apache.org and flink playgrounds respectively are the >> best places to host this content. >> >> On Thu, Apr 9, 2020 at 2:56 PM Niels Basjes wrote: >> >>> Hi, >>> >>> Sounds like a very nice thing to have as part of the pro

Re: [PROPOSAL] Contribute training materials to Apache Flink

gt; > One thing the community should be aware of is that we also need to > maintain the training material. Maybe you could share with us how much > effort this usually is when updating the training material for a new Flink > version, David? > > Cheers, > Till > > On

[DISCUSS] Integration of training materials into Apache Flink

Thank you all for the very positive response to our proposal to contribute the training materials that have been at training.ververica.com to the Apache Flink project. Now I’d like to begin the more detailed discussion of how to go about this. In that earlier thread I mentioned that we were thinki

Re: [DISCUSS] Integration of training materials into Apache Flink

d salvage anything worth keeping, like the glossary. > > Seth > > On Wed, Apr 15, 2020 at 2:02 PM David Anderson > wrote: > > > Thank you all for the very positive response to our proposal to > contribute > > the training materials that have been at training.ververic

Re: [DISCUSS] Integration of training materials into Apache Flink

@Robert, do you think we could create a new flink-training component in Jira for tracking tickets related to the flink-training content? David On Thu, Apr 16, 2020 at 10:54 AM David Anderson wrote: > > I am happy to get the repo created for you. > > Thank you, @seth. I think we ar

Re: [DISCUSS] Integration of training materials into Apache Flink

> > How about calling the component "Documentation / Training"? Sounds good. Thanks! *David Anderson* | Training Coordinator Follow us @VervericaData -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time On Sun, Apr 19, 2020 at

Re: [DISCUSS] FLIP-126: Unify (and separate) Watermark Assigners

Overall I like this proposal; thanks for bringing it forward, Aljoscha. I also like the idea of making the Watermark generator a rich function -- this should make it more straightforward to implement smarter watermark generators. Eg, one that uses state to keep statistics about the actual out-of-o

Re: [DISCUSS] Move docker development into versioned branches

Makes sense to me. I think this would align well enough with user expectations, and be more straightforward. David On Thu, Apr 23, 2020 at 1:23 PM Chesnay Schepler wrote: > Hello everyone, > > Currently, all development in docker-flink occurs on the master branch, > for all releases at once. Th

Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type

+1 for RAW. I agree that this is clearer than OPAQUE (which I initially proposed). On Mon, Oct 21, 2019 at 10:33 AM Jark Wu wrote: > > +1 to rename ANY. > > I don't have strong opinion on the new name. I think "OPAQUE" is fine, > because it is introduced in IBM Informix and Oracle. > In Informi

Re: [DISCUSS] Introduce a location-oriented two-stage query mechanism toimprove the queryable state.

I've encountered a number of Flink users who considered using queryable state, but after investigation, decided not to. The reasons have been: (1) The current interface (point queries fetching state for one key) is too limiting. What some folks really want/need is the ability to execute SQL-querie

Re: [VOTE] Release 1.10.0, release candidate #1

+1 (non-binding) No blockers, but I did run into a couple of things. I upgraded flink-training-exercises to 1.10; after minor adjustments, all the tests pass. With Java 11 I ran into WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.fli

Re: [DISCUSS] Feature freeze for Flink 1.7

tic, too early, too late) and > >> which features/issues do you think should get priority? > >> > >> Since every release needs a release manager, I would also volunteer for > it. > >> > >> Cheers, > >> Till > >> > > -- *David Anderson* | Training Coordinator | data Artisans -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time

Re: [DISCUSS] Clean up and reorganize the JIRA components

> > We can also try to assign documentation components to both "Documentation" > and the affected component, such as "Runtime / Metrics". This is critical for anyone trying to keep an eye on the documentation as a whole -- e.g., ensuring that it remains readable, is well-organized, and is being t

Re: [DISCUSS] Releasing Flink 1.11.2

I think it's worth considering whether we can get this bugfix included in 1.11.2: - FLINK-19109 Split Reader eats chained periodic watermarks There is a PR, but it's still a work in progress. Cc'ing Roman, who has been working on this. Regards, David On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu wro

Re: [VOTE] FLIP-131: Consolidate the user-facing Dataflow SDKs/APIs (and deprecate the DataSet API)

+1 On Thu, Sep 3, 2020 at 3:06 PM Piotr Nowojski wrote: > +1 > > czw., 3 wrz 2020 o 10:08 Guowei Ma napisał(a): > > > +1 > > Looking forward to having a unified datastream api. > > Best, > > Guowei > > > > > > On Thu, Sep 3, 2020 at 3:46 PM Dawid Wysakowicz > > wrote: > > > > > +1 > > > > > >

Re: [DISCUSS] FLIP-134: DataStream Semantics for Bounded Input

Having just re-read FLIP-134, I think it mostly makes sense, though I'm not exactly looking forward to figuring out how to explain it without making it seem overly complicated. A few points: I'm a bit confused by the discussion around custom window Triggers. Yes, I agree that complex, mixed Trigg

Re: [VOTE] Release 1.11.2, release candidate #1

+1 Checks: - Verified that the fix for FLINK-19109 solves the problem I reported, running against the maven artifacts On Thu, Sep 10, 2020 at 9:04 AM Zhu Zhu wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 1.11.2, > as follows: > [ ] +1, Approve th

Re: [ANNOUNCE] New Apache Flink Committer - Niels Basjes

Congratulations! --David On Mon, Sep 14, 2020 at 8:24 PM Arvid Heise wrote: > Congrats Niels! > > On Mon, Sep 14, 2020 at 4:04 PM Benchao Li wrote: > > > Congratulations! > > > > Xingbo Huang 于2020年9月14日周一 下午9:36写道： > > > > > Congratulations! > > > > > > Best, > > > Xingbo > > > > > > Dian Fu

Re: [ANNOUNCE] New Apache Flink Committer - Igal Shilman

Congrats, Igal! Well deserved. David On Tue, Sep 15, 2020 at 10:14 AM Tzu-Li (Gordon) Tai wrote: > Hi all, > > It's great seeing many new Flink committers recently, and to add to that > I'd like to announce one more new committer: Igal Shilman! > > Igal has been a long time member of the commun

Re: [ANNOUNCE] New Apache Flink Committer - Arvid Heise

Congratulations, Arvid! Well deserved. Best, David On Tue, Sep 15, 2020 at 10:23 AM Paul Lam wrote: > Congrats, Arvid! > > Best, > Paul Lam > > > 2020年9月15日 15:29，Jingsong Li 写道： > > > > Congratulations Arvid ! > > > > Best, > > Jingsong > > > > On Tue, Sep 15, 2020 at 3:27 PM Dawid Wysakowicz

Re: [VOTE] FLIP-142: Disentangle StateBackends from Checkpointing

+1 (binding) David On Tue, Sep 15, 2020 at 10:25 AM Tzu-Li (Gordon) Tai wrote: > +1 (binding) > > On Tue, Sep 15, 2020 at 3:26 PM Aljoscha Krettek > wrote: > > > +1 (binding) > > > > Aljoscha > > >

Re: KeyedCoProcessFunction, processElement1, processElement2, onTimer timeout

Mazen, questions like this are better suited to the user mailing list. FYI, this is also being discussed on stackoverflow: https://stackoverflow.com/questions/63902457/flink-timeout-using-keyedcoprocessfunction-and-order-of-reading-for-flinkkafkaco Regards, David On Wed, Sep 16, 2020 at 9:41 AM

Re: [DISCUSS] FLIP-134: DataStream Semantics for Bounded Input

for the thoughtful comments! I'll try and address them inline > below. I'm hoping to start a VOTE thread soon if there are no other > comments by the end of today. > > On 10.09.20 15:40, David Anderson wrote: > > Having just re-read FLIP-134, I think it mostly makes sense, t

Re: Re: [VOTE] FLIP-134: Batch execution for the DataStream API

+1 Thanks for the discussion. Looking forward to seeing this all come to pass! David On Wed, Sep 16, 2020 at 9:28 AM Guowei Ma wrote: > +1 (non-binding) > > Best, > Guowei > > > On Wed, Sep 16, 2020 at 3:03 PM Yun Gao > wrote: > > > +1 (non-binding) > > > > Very thanks for bring this up! And

Re: Need help in setting up flink 1.10

Ravi, Note that questions like this are better suited for the user mailing list. According to [1], google cloud storage is supported under the gcs: url scheme. Also, since Flink 1.10, most filesystems must be loaded as plugins, rather than from the lib directory [2]. I don't have experience with

Re: [DISCUSS] Remove flink-connector-filesystem module.

I think the pertinent question is whether there are interesting cases where the BucketingSink is still a better choice. One case I'm not sure about is the situation described in docs for the StreamingFileSink under Important Note 2 [1]: ... upon normal termination of a job, the last in-progres

Re: [DISCUSS] Remove flink-connector-filesystem module.

tdown Handling -- and in fact, the StreamingFileSink is mentioned in that FLIP as a motivating use case. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-46%3A+Graceful+Shutdown+Handling+by+UDFs On Tue, Oct 13, 2020 at 1:01 PM Aljoscha Krettek wrote: > On 13.10.20 11:18, David Anderson wrote:

Re: [DISCUSS] Support KeyedSortedMapState in DataStream API

I'm very interested in this topic, and have even done some prototyping of solution 1 -- limiting the key type to Long -- which Nico Kruber and I called TemporalState in our prototype. I look forward to sharing what we learned, and to discussing this further, but I am completely overwhelmed with Fl

Re: [DISCUSS] Make SQL docs Blink only

I agree -- I think separating out the legacy planner info should make things clearer for everyone, and then some day we can simply drop it. Plus, doing it now will make it easier to make improvements to the docs going forward. David On Tue, Dec 8, 2020 at 4:38 PM Timo Walther wrote: > Hi Seth,

Re: [DISCUSS] Release Flink 1.16.1

Martijn, Thank you for bringing this up. From my (admittedly narrow) perspective, I'd like to see a release sooner rather than later, as there's an already merged bug fix I'd like to get released. David On Thu, Dec 15, 2022 at 1:53 PM Martijn Visser wrote: > Hi everyone, > > I would like to

Re: [VOTE] Update Flink's Scala 2.12 support from 2.12.7 to 2.12.16

+1 (binding) On Fri, Dec 16, 2022 at 12:22 PM Martijn Visser wrote: > Hi all, > > I'm bumping this old vote thread once more. > > If we want to add Java 17 support at some point, we will need to update > our Scala 2.12 version (see > https://issues.apache.org/jira/browse/FLINK-25000). As expla

Re: [DISCUSS] Extending the feature freezing date of Flink 1.17

I'm also in favor of extending the feature freeze to Jan 31st. David On Thu, Dec 29, 2022 at 9:01 AM Leonard Xu wrote: > Thanks Qingsheng for the proposal, the pandemic has really impacted > development schedules. > > Jan 31st makes sense to me. > > > Best, > Leonard > >

Re: [DISCUSS] Incorporate performance regression monitoring into routine process

+1 I don't have anything substantive to add, but I want to express how pleased I am to see this conversation happening. David On Thu, Feb 2, 2023 at 5:09 AM Martijn Visser wrote: > Hi all, > > +1 for the overall proposal. My feedback matches with what Matthias > has already provided earlier. >

Re: Unit Testing onTimer() event function with TestHarness - onTimer() not being called

1. The timestamp passed to testHarness.processElement should be the timestamp that would have been extracted from the element by the timestamp extractor in your watermark strategy. 2. Your tests should call testHarness.processWatermark and pass in the watermark(s) you want to work with. processBr

what happened to the images in FLIP-24: SQL Client?

Does anyone know what happened to the diagrams that used to be in FLIP-24: SQL Client? The last time I looked at this FLIP -- a few weeks ago -- there were architecture diagrams for Gateway Mode and Embedded Mode, but now those images are missing. David

[DISCUSS] FLIP-220: Temporal State

Greetings, Flink developers. I would like to open up a discussion of a proposal [1] to add a new kind of state to Flink. The goal here is to optimize a fairly common pattern, which is using MapState> to store lists of events associated with timestamps. This pattern is used internally in quite a

Re: [DISCUSS] FLIP-220: Temporal State

ser look. > > > Besides, I have one question that can we support expose the namespace > in > > > the different state type not limited to `TemporalState`. By this, user > > can > > > specify the namespace > > > and the TemporalState is one of the special c

Re: [DISCUSS] FLIP-220: Temporal State

; the namespace. I think it will be more extendable. > What do you think about this ? > > Best, > Aitozi. > > David Anderson 于2022年4月11日周一 20:54写道： > > > Greetings, Flink developers. > > > > I would like to open up a discussion of a proposal [1] to add a n

Re: [DISCUSS] FLIP-220: Temporal State

rent state type not limited to `TemporalState`. By this, user can > specify the namespace > and the TemporalState is one of the special case that it use timestamp as > the namespace. I think it will be more extendable. > What do you think about this ? > > Best, > Aitozi. &

Re: [Discuss] Creating an Apache Flink slack workspace

I have mixed feelings about this. I have been rather visible on stack overflow, and as a result I get a lot of DMs asking for help. I enjoy helping, but want to do it on a platform where the responses can be searched and shared. It is currently the case that good questions on stack overflow frequ

Re: [Discuss] Creating an Apache Flink slack workspace

pared >>> >> to the user-zh@ ML, which I'd attribute to the improvement of >>> interaction >>> >> experiences. Admittedly, there are questions being repeatedly asked & >>> >> answered, but TBH I don't think that compares to the benefit

Re: Request for Review: FLINK-27507 and FLINK-27509

I've taken care of this. David On Sun, May 22, 2022 at 4:12 AM Shubham Bansal wrote: > Hi Everyone, > > I am not sure who to reach out for the reviews of these changesets, so I > am putting this on the mailing list here. > > I have raised the review for > FLINK-27507 - https://github.com/apache

Re: [DISCUSS] Deprecate SourceFunction APIs

I'm in favor of this, but I think we need to make it easier to implement data generators and test sources. As things stand in 1.15, unless you can be satisfied with using a NumberSequenceSource followed by a map, things get quite complicated. I looked into reworking the data generators used in the

Re: [DISCUSS] Deprecate SourceFunction APIs

a new Source is too high now. > > > > > > > > > > > > > > > > Ingo Bürk 于2022年6月5日 周日05:32写道： > > > > > > > > > I +1 everything David said. The new Source API raised the > complexity > > > > > sig

[DISCUSS] Releasing 1.15.1

I would like to start a discussion on releasing 1.15.1. Flink 1.15 was released on the 5th of May [1] and so far 43 issues have been resolved, including several user-facing issues with blocker and critical priorities [2]. (The recent problem with FileSink rolling policies not working properly in 1.

Re: [DISCUSS] Deprecate SourceFunction APIs

all > > > > >>>>>>> boundedness. Do you think something like this could be built > > from > > > > >>> the > > > > >>>>>> same > > > > >>>>>>> pieces? > > > > >>

Re: [DISCUSS] Releasing 1.15.1

for volunteering to manage the release. > > > > > > > > > > Best, > > > > > Jingsong > > > > > > > > > > On Wed, Jun 8, 2022 at 6:21 PM Jark Wu wrote: > > > > >> Hi David, thank you for driving the relea

Re: [DISCUSS ] Make state.backend.incremental as true by default

Thank you for bringing this up! +1 On Mon, Jun 13, 2022 at 1:48 PM Rui Fan <1996fan...@gmail.com> wrote: > Strongly +1 > > Best, > Rui Fan > > On Mon, Jun 13, 2022 at 7:35 PM Martijn Visser > wrote: > > > > BTW, from my knowledge, nothing would happen for HashMapStateBackend, > > which does no

Re: [DISCUSS] Drop Gelly

Most of the inquiries I've had about Gelly in recent memory have been from folks looking for a streaming solution, and it's only been a handful. +1 for dropping Gelly David On Mon, Jan 3, 2022 at 2:41 PM Till Rohrmann wrote: > I haven't seen any changes or requests to/for Gelly in ages. Hence,

Re: [DISCUSS] FLIP-203: Incremental savepoints

> I have a very similar question to State Processor API. Is it the same scenario in this case? > Should it also be working with checkpoints but might be just untested? I have used the State Processor API with aligned, full checkpoints. There it has worked just fine. David On Thu, Jan 13, 2022 at

Re: [VOTE] FLIP-203: Incremental savepoints

+1 (non-binding) I'm pleased to see this significant improvement coming along, as well as the effort made in the FLIP to document what is and isn't supported (and where ??? remain). On Wed, Jan 26, 2022 at 10:58 AM Yu Li wrote: > +1 (binding) > > Thanks for driving this Piotr! Just one more (be

Re: Stack Overflow Question - Deserialization schema for multiple topics

For questions like this one, please address them to either Stack Overflow or the user mailing list, but not both at once. Those two forums are appropriate places to get help with using Flink's APIs. And once you've asked a question, please allow some days for folks to respond before trying again.

Re: [DISCUSS] Deprecate/remove Twitter connector

I agree. The Twitter connector is used in a few (unofficial) tutorials, so if we remove it that will make it more difficult for those tutorials to be maintained. On the other hand, if I recall correctly, that connector uses V1 of the Twitter API, which has been deprecated, so it's really not very

Re: [VOTE] Remove Twitter connector

+1 On Mon, Jan 31, 2022 at 11:47 AM Martijn Visser wrote: > Hi everyone, > > I would like to open up a vote to remove the Twitter connector in Flink > 1.15. This was brought up previously for a discussion [1]. > > The vote will last for at least 72 hours, and will be accepted by > a consensus of

status of Apple Silicon (M1) as Flink dev platform?

What's the current status of using the Apple Silicon (M1) platform for Flink development? Have we reached the point where everything "just works", or do there remain lingering annoyances (or worse)? In the past, I've seen reports of issues involving, e.g., RocksDB, nodejs, protobuf, and pyflink. L

Re: [DISCUSS] Enable scala formatting check

+1 For flink-training we initially tried cloning the scalastyle setup from flink, but we decided to use spotless + scalafmt instead. David On Mon, Mar 7, 2022 at 1:12 PM Timo Walther wrote: > Big +1 > > This will improve the contribution experience. Even though we stopped > adding more Scala c

Re: [DISCUSS] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

I've spent a few hours digging into this with Seth, and can report that this makes working on the docs much less of a chore. +1 (with enthusiasm) Best, David On Thu, Jan 14, 2021 at 1:34 PM Kostas Kloudas wrote: > +1 for moving to Hugo. > > Cheers, > Kostas > > On Thu, Jan 14, 2021 at 1:27 PM

Re: [Vote] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

+1 David On Tue, Jan 19, 2021 at 5:28 AM Forward Xu wrote: > +1 > > Dian Fu 于2021年1月19日周二 上午11:40写道： > > > +1 > > > > > 在 2021年1月19日，上午11:34，Jark Wu 写道： > > > > > > +1 > > > > > > On Tue, 19 Jan 2021 at 01:59, Till Rohrmann > > wrote: > > > > > >> +1, > > >> > > >> Cheers, > > >> Till > > >>

Re: [ANNOUNCE] Welcome Roman Khachatryan a new Apache Flink Committer

Congratulations, Roman! Glad to have you onboard!! David On Thu, Feb 18, 2021 at 10:51 AM Congxian Qiu wrote: > Congratulations, Roman > Best, > Congxian > > > Leonard Xu 于2021年2月18日周四 下午1:47写道： > > > Congrats Roman! > > > > Best, > > Leonard > > > > > 在 2021年2月18日，11:10，Yu Li 写道： > > > > > >

Re: [DISCUSS] FLIP-165: Operator's Flame Graphs

This is going to make performance analysis and optimization much more accessible. I can't wait to include this in our training courses. +1 Seth suggested putting the docs for this feature under Operations/Monitoring, but there's already a page in the docs under Operations/Debugging for Applicatio

Re: [VOTE] FLIP-151: Incremental snapshots for heap-based state backend

+1 (non-binding) On Mon, Mar 1, 2021 at 10:12 AM Roman Khachatryan wrote: > Hi everyone, > > since the discussion [1] about FLIP-151 [2] seems to have reached a > consensus, I'd like to start a formal vote for the FLIP. > > Please vote +1 to approve the FLIP, or -1 with a comment. The vote will

Re: [VOTE] Release 1.13.0, release candidate #2

+1 (non-binding) Checks: - I built from source, successfully. - I tested the new backpressure metrics and UI. I found one non-critical bug that's been around for years, and for which a fix has already been merged for 1.13.1 (https://issues.apache.org/jira/browse/FLINK-22489

trying (and failing) to update pyflink-walkthrough for Flink 1.13

I've been trying to upgrade the pyflink-walkthrough to Flink 1.13.1, but without any success. Unless I give it a lot of resources the data generator times out trying to connect to Kafka. If I give it 6 cores and 11GB (which is about all I can offer it) it does manage to connect, but then fails try

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

I am hearing quite often from users who are struggling to manage memory usage, and these are all users using RocksDB. While I don't know for certain that RocksDB is the cause in every case, from my perspective, getting the better memory stability of version 6.20 in place is critical. Regards, Davi

Were Bundles meant to be internal?

Most of the table/bundle related classes such as MapBundleFunction, MapBundleOperator, and CountBundleTrigger aren't marked as either @Internal or @Public. What was the intention? I ask because I'm starting to see some interest in using them for implementing pre-aggregation via the DataStream API,

Re: [DISCUSS] Conventions on assertions to use in tests

For what it's worth, I recently rewrote all of the tests in flink-training to use assertj, removing a mixture of junit4 assertions and hamcrest in the process. I chose assertj because I found it to be more expressive and made the tests more readable. +1 from me David On Fri, Nov 12, 2021 at 10:0

Re: [DISCUSS] Releasing 1.15.1

I'm now thinking we should delay 1.15.1 long enough to see if we can include a fix for FLINK-28060 [1], which is a serious regression affecting several Kafka users. [1] https://issues.apache.org/jira/browse/FLINK-28060 On Fri, Jun 10, 2022 at 12:15 PM David Anderson wrote: > Since no

Re: [DISCUSS] Releasing 1.15.1

er 1.15.1 is done, to see how it behaves on the CI for the next few weeks, and plan a quick Flink 1.15.2 release (most likely something like a month later). [1] https://issues.apache.org/jira/browse/FLINK-28060 Best, David On Wed, Jun 15, 2022 at 11:37 AM David Anderson wrote: > I

Re: [DISCUSS] FLIP-238: Introduce FLIP-27-based Data Generator Source

I'm very happy with this. +1 A lot of SourceFunction implementations used in demos/POC implementations include a call to sleep(), so adding rate limiting is a good idea, in my opinion. Best, David On Mon, Jun 20, 2022 at 10:10 AM Qingsheng Ren wrote: > Hi Alexander, > > Thanks for creating thi

[VOTE] Release 1.15.1, release candidate #1

Hi everyone, Please review and vote on release candidate #1 for version 1.15.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the of

[RESULT][VOTE] Release 1.15.1, release candidate #1

I am pleased to announce that we have approved this release candidate. There are 7 approving votes, 4 of which are binding: - Chesnay Schepler (+1 binding) - Xingbo Huang (+1 non-binding) - Qingsheng Ren (+1 non-binding) - Robert Metzger (+1 binding) - Konstantin Knauf (+1 binding) - Jingsong Li (

[ANNOUNCE] Apache Flink 1.15.1 released

available in Jira: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12351546 We would like to thank all contributors of the Apache Flink community who made this release possible! Regards, David Anderson

Re: [DISCUSS] FLIP-251: Support collecting arbitrary number of streams

I've found that with our current tooling it's frustrating to try to write good end-to-end tests for real-world jobs with multiple sinks. DataStream#executeAndCollect() is okay for simple pipelines with one sink, but in my opinion we do need something like FLIP-251. The proposed interface looks goo

Re: [VOTE] FLIP-251: Support collecting arbitrary number of streams

+1 Thank you Chesnay. On Tue, Jul 19, 2022 at 3:09 PM Alexander Fedulov wrote: > +1 > Looking forward to using the API to simplify tests setups. > > Best, > Alexander Fedulov > > On Tue, Jul 19, 2022 at 2:31 PM Martijn Visser > wrote: > > > Thanks for creating the FLIP and opening the vote Che

Re: [DISCUSS] Bump Kafka to 3.2.1 for 1.16.0

I'm in favor of adopting this fix in 1.16.0. +1 On Tue, Aug 9, 2022 at 7:13 AM tison wrote: > +1 > > This looks reasonable. > > Best, > tison. > > > Thomas Weise 于2022年8月9日周二 21:33写道： > > > +1 for bumping the Kafka dependency. > > > > Flink X.Y.0 releases require thorough testing, so consideri

Re: [VOTE] FLIP-265 Deprecate and remove Scala API support

+1 On Mon, Oct 17, 2022 at 3:39 PM Martijn Visser wrote: > Hi everyone, > > I'm hereby opening a vote for FLIP-265 Deprecate and remove Scala API > support. The related discussion can be found here [1]. > > Voting will be open for at least 72 hours. > > Best regards, > > Martijn > https://twit

Re: [VOTE] Drop Gelly

+1 On Wed, Oct 12, 2022 at 10:59 PM Martijn Visser wrote: > Hi everyone, > > I would like to open a vote for dropping Gelly, which was discussed a long > time ago but never put to a vote [1]. > > Voting will be open for at least 72 hours. > > Best regards, > > Martijn > https://twitter.com/Marti

Re: [DISCUSS] Remove FlinkKafkaConsumer and FlinkKafkaProducer in the master for 1.17 release

> > [H]ow one can migrate from the > FlinkKafkaConsumer/FlinkKafkaProducer to KafkaSource/KafkaSink, while > preserving exactly-once guarantees etc? The responses from Fabian Paul in [1] and [2] address the question of how to handle the migration in terms of managing the state (where the short an

Re: [DISCUSS] Remove FlinkKafkaConsumer and FlinkKafkaProducer in the master for 1.17 release

> > For the partition > idleness problem could you elaborate more about it? I assume both > FlinkKafkaConsumer and KafkaSource need a WatermarkStrategy to decide > whether to mark the partition as idle. As a matter of fact, no, that's not the case -- which is why I mentioned it. The FlinkKafkaCo

Re: [DISCUSS]Refactor flink-jdbc connector structure

I'm very happy to see the jdbc connector being normalized in this way. +1 from me. David On Thu, Apr 30, 2020 at 2:14 PM Timo Walther wrote: > Hi Leonard, > > this sounds like a nice refactoring for consistency. +1 from my side. > > However, I'm not sure how much backwards compatibility is requ

  1   2   3   >