[Discuss][FLIP-42] Documentation Style Guide

to hearing the community's feedback. Seth Wiesman [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-42%3A+Rework+Flink+Documentation [2] https://github.com/apache/flink-web/pull/240

Re: [DISCUSS] Flink Docker Playgrounds

Hey Fabian, I support option 1. As per FLIP-42, playgrounds are going to become core to flinks getting started experience and I believe it is worth the effort to get this right. - As you mentioned, we may (and in my opinion definitely will) add more images in the future. Setting up an integr

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

I would prefer ‘lit()’ over ‘val()’ since val is a keyword in Scala. Assuming the intention is to make the dsl ergonomic for Scala developers. Seth > On Aug 28, 2019, at 7:58 AM, Timo Walther wrote: > > Hi David, > > thanks for your feedback. I was also skeptical about 1 char method names,

Re: Checkpoint metrics.

Great timing, I just debugged this on Monday. E2e time is checkpoint coordinator to checkpoint coordinator, so it includes RPC to the source and RPC from the operator back for the JM. Seth > On Sep 11, 2019, at 6:17 PM, Jamie Grier wrote: > > Hey all, > > I need to make sense of this behav

Re: Flink dev blog

+1 on the idea. My only request would be they are clearly marked as being about internals / for advanced users to not give typical users the wrong impression about how much they need to understand to use Flink. Nico's network stack blog post does this well[1]. Seth [1] https://flink.apache.org/2

Re: Flink dev blog

> >>>>> And we would also be happy to contribute some contents on the > newest > > >>>>> efforts from our team. > > >>>>> Potential topics: > > >>>>> - Memory configuration > > >>>>> - Active Kuber

Re: [DISCUSS] Update on Flink Stateful Functions & what are the next steps?

+1 to release. Stateful Functions has a strong set of core features, and a released version will help drive adoption which will in turn help shape feature development. Seth On Mon, Mar 9, 2020 at 6:59 AM Stephan Ewen wrote: > Thanks, Gordon for this update. > > I think it would be great to do

Re: [DISCUSS] Drop Bucketing Sink

I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we want to drop the bucketing sink. Seth On Thu, Mar 12, 2020 at 10:53 AM David Anderson wrote: > The BucketingSink is still somewhat widely used, I think in part because of > shortcomings in the StreamingFileSink. > > I wo

Re: [DISCUSS] Drop Bucketing Sink

Sorry, I meant FLIP-46. Seth On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman wrote: > I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we > want to drop the bucketing sink. > > Seth > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson > wrote: >

Re: [Dev Blog] Migrating Flink's CI Infrastructure from Travis CI to Azure Pipelines

Very interesting! No questions but thank you for taking the initiative to put out the first dev blog. Seth > On Mar 23, 2020, at 5:14 AM, Robert Metzger wrote: > > Hi all, > > I have just published the first post to the dev blog: > https://cwiki.apache.org/confluence/display/FLINK/2020/03/22

Re: [VOTE] Apache Flink Stateful Functions Release 2.0.0, release candidate #6

+1 (non-binding) legal / source - checked sources for binary files - checked license headers functional - built from source (mvn clean verify -Prun-e2e-tests) - built python sdk and ran tests - ran examples - deployed mixed python / java application on k8s with checkpointing. Failed TM's and watc

Re: [DISCUSS] FLIP-124: Add open/close and Collector to (De)SerializationSchema

I would be in favor of buffering data outside of the checkpoint lock. In my experience, serialization is always the biggest performance killer in user code and I have a hard time believing in practice that anyone is going to buffer so many records that is causes real memory concerns. To add to Tim

Re: [ANNOUNCE] New Flink committer: Seth Wiesman

- >> From:tison >> Send Time:2020 Apr. 7 (Tue.) 19:26 >> To:dev >> Subject:Re: [ANNOUNCE] New Flink committer: Seth Wiesman >> >> Congratulations, Seth! >> >> Best, >> tison. >> >>

Re: [PROPOSAL] Contribute training materials to Apache Flink

Hi David, +1 to add to the project. I agree that flink.apache.org and flink playgrounds respectively are the best places to host this content. On Thu, Apr 9, 2020 at 2:56 PM Niels Basjes wrote: > Hi, > > Sounds like a very nice thing to have as part of the project ecosystem. > > Niels > > On T

Re: [DISCUSS] Integration of training materials into Apache Flink

Hi David, I am happy to get the repo created for you. If we go with the documentation (vs flink.apache.org) do you think we should remove any of the existing content? There is already a getting started section with quickstarts and walkthroughs and a concepts section. In particular, the concepts s

Re: [DISCUSS] Integration of training materials into Apache Flink

Aljoscha, I know you've been working on the docs recently. Are you working on the concepts section? I don't want to just drop it if someone has work in the pipeline. Seth On Wed, Apr 15, 2020 at 2:19 PM Seth Wiesman wrote: > Hi David, > > I am happy to get the repo created

Re: [DISCUSS] [FLINK-16824] Creating Temporal Table Function via DDL

I really like the TEMPORAL keyword, I find it very intuitive. The down side of this approach would be that an additional preprocessing > step would not be possible anymore because there is no preceding view. > Yes and no. My understanding is we are not talking about making any changes to how tem

Re: [DISCUSS] Integration of training materials into Apache Flink

with what Aljoscha is doing there. >> But yes, there is room for improvement in this part of the docs, so I'm >> expecting to be able to help with that. >> >> David >> >> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman wrote: >> >>> Hi David, >

Re: [VOTE] Accept Stateful Functions into Apache Flink

+1 (non-binding) Seth > On Oct 23, 2019, at 9:31 PM, Jingsong Li wrote: > > +1 (non-binding) > > Best, > Jingsong Lee > >> On Wed, Oct 23, 2019 at 9:02 PM Yu Li wrote: >> >> +1 (non-binding) >> >> Best Regards, >> Yu >> >> >>> On Wed, 23 Oct 2019 at 16:56, Haibo Sun wrote: >>> >>> +1

[DISCUSS] Drop vendor specific deployment documentation.

r question of which vendor products should be included and which should not. That is why I would like to suggest dropping these pages and referring users to vendor maintained documentation whenever they are using one of these services. Seth Wiesman

Re: [DISCUSS] Drop vendor specific deployment documentation.

from Google Analytics on how > > popular > > > >>> those pages are? If they are somewhat popular, I would prefer to > "fix > > > >> them" > > > >>> to be good starting points for users in those environments > (probably > >

[DISCUSS] Flink Docs Style Guide Review

Hi All! A while back, Marta opened a PR to create a documentation style guide as part of FLIP-42[1][2]. Unfortunately, the review stalled out as everyone involved got busy with Flink Forward Europe. Since we are approaching the final stretch for Flink 1.10 and I expect to see an influx in document

Re: [DISCUSS] Drop vendor specific deployment documentation.

t; > On Thu, Dec 5, 2019 at 6:28 PM Seth Wiesman wrote: > > > One option would be to do exactly that, but then I feel like we are > > committing to tracking changes on those systems and I just don't know how > > feasible that is. > > > > I don't think tha

Re: [DISCUSS] have separate Flink distributions with built-in Hive dependencies

I'm also -1 on separate builds. What about publishing convenience jars that contain the dependencies for each version? For example, there could be a flink-hive-1.2.1-uber.jar that users could just add to their lib folder that contains all the necessary dependencies to connect to that hive version.

[DISCUSS] Flink docs vendor table

This discussion is a follow up to the previous thread on dropping vendor-specific documentation[1]. The conversation ended unresolved on the question of what we should provide on the Apache Flink docs. The consensus seemed to be moving towards offering a table with links to 3rd parties. After an o

Re: [DISCUSS] Flink docs vendor table

offering > > > links to 3rd party integrations. > > > > > > Cheers, > > > Till > > > > > > On Mon, Dec 16, 2019 at 6:04 PM Seth Wiesman > > wrote: > > > > > > > This discussion is a follow up to the previous thread on dro

Re: [DISCUSS] Flink docs vendor table

ses software (like Ververica Platform). This > > would be beneficial, but on the other hand the categories/terms (managed, > > hosted, "serverless", self-managed) are not so well-defined in my > > experience. > > > > On Tue, Dec 17, 2019 at 10:46 PM Seth W

Re: [DISCUSS] Flink docs vendor table

as not a good idea. > >> On Fri, Dec 20, 2019 at 5:35 AM Bowen Li wrote: >> >> Really cool. I especially like the list of tags on "Ververica Platform"! >> >> BTW, why is "Ververica Platform" placed at the last? I won't feel bothere

Re: [DISCUSS] Support Interactive Programming in Flink Table API

I spoke to Piotr a little bit offline and I wanted to comment with a summary of our discussion and what I believe is most intuitive cache model from a users perspective. (I am making up some class names here, not looking to bike shed feel free to change the names how ever you see fit). A cac

Re: Flink solution to active - active Multi site cloud data ingestion

an aggregation result that will reflect all messages transmitted by > specific device X. > > Are there any best practices to handle multi-site ingestion? > > Any idea how to handle the scenario above? > > Thanks in advance. > > -- Seth Wiesman | Solutions A

Re: Flink solution to active - active Multi site cloud data ingestion

In the future, these kinds of questions are more appropriate for the user mailing list (u...@flink.apache.org). Dev is for internal Flink development. On Wed, May 15, 2019 at 12:06 PM Seth Wiesman wrote: > Hi Gregory, > > The easiest solution would be to include the site in your key s

[DISCUSS] FLIP-42: Savepoint Connector

+Connector <https://cwiki.apache.org/confluence/display/FLINK/FLIP-41%3A+Unify+Keyed+State+Snapshot+Binary+Format+for+Savepoints> . -- Seth Wiesman

[Discuss] FLIP-43: Savepoint Connector

Hey Everyone! Gordon and I have been discussing adding a savepoint connector to flink for reading, writing, and modifying savepoints. This is useful for: * Analyzing state for interesting patterns * Troubleshooting or auditing jobs by checking for discrepancies in state * Bootstrapping state fo

[Discuss] FLIP-43: Savepoint Connector

Hey Everyone! ​ Gordon and I have been discussing adding a savepoint connector to flink for reading, writing and modifying savepoints. ​ This is useful for: ​ Analyzing state for interesting patterns Troubleshooting or auditing jobs by checking for discrepancies in state Bootstrapping

Re: [Discuss] FLIP-43: Savepoint Connector

t; >> Glad to see this FLIP, big +1 for this feature! >> >> Best, >> Vino >> >> Seth Wiesman 于2019年5月30日周四 上午7:14写道： >> >>> Hey Everyone! >>> ​ >>> Gordon and I have been discussing adding a savepoint connector to flin

Re: [Discuss] FLIP-43: Savepoint Connector

>>> >>> On May 29, 2019, at 7:37 PM, Paul Lam wrote: >>> >>> Hi Seth, >>> >>> +1 from my side. >>> >>> I was wondering if we can add a reader method to provide a full view of >>> the states instead of the state of a specific

Re: [Discuss] FLIP-43: Savepoint Connector

tly >> asking for this and it would be good to know what to answer them. >> >> Piotrek >> >>> On 29 May 2019, at 17:51, Seth Wiesman wrote: >>> >>> Hey Everyone! >>> >>> Gordon and I have been discussing adding a savepoint

Re: [Discuss] FLIP-43: Savepoint Connector

e this happen. The question is - would > it be possible to design this feature so that writing the savepoint from > different runner would be possible? > > Cheers, > > Jan > >> On 5/30/19 1:14 AM, Seth Wiesman wrote: >> Hey Everyone! >> ​ >> Gordon

Re: [Discuss] FLIP-43: Savepoint Connector

> configuration required, like just opening SQL CLI with just maybe a pointer >> to savepoint id? >> >> Side question, are you aiming with this for 1.9 release? 1.10? Or you do >> not have a specific time frame? I’m asking since users are quite frequently >> ask

Re: [Discuss] FLIP-43: Savepoint Connector

ibrary, of course). > > Jan > >> On 5/31/19 1:17 PM, Seth Wiesman wrote: >> @Konstantin agreed, that was a large impotence for this feature. Also I am >> happy to change the name to something that better describes the feature set. >> >> @Lan >>

Re: [Discuss] FLIP-43: Savepoint Connector

design, these are just ideas of "what could > be possible", I was just wondering if this FLIP can make some first steps > towards this. > > Many thanks for comments, > > Jan > >> On 5/31/19 8:12 PM, Seth Wiesman wrote: >> @Jan Gotcha, >> &

Re: [Discuss] FLIP-43: Savepoint Connector

It seems like a recurring piece of feedback was a different name. I’d like to propose moving the functionality to the libraries module and naming this the State Processing API. Seth > On May 31, 2019, at 3:47 PM, Seth Wiesman wrote: > > The SavepointOutputFormat only write

[Proposal] RichFsSinkFunction

-using-manifest.html . I have below an example gist of what this looks like to use. Example: https://gist.github.com/sjwiesman/fc99c64f44a93cfc9c7aa62c070a9358 [cid:image001.png@01D2763A.4A98E300]<https://www.mediamath.com/mailto> Seth Wiesman | Data Engineer
 4 World Trade Center, 45th

Re: [DISCUSS] FLIP-140: Introduce bounded style execution for keyed streams

There is already an implicit assumption the TypeSerializer for keys is stable/deterministic, RocksDB compares keys using their serialized byte strings. I think this is a non-issue (or at least it's not changing the status quo). On Fri, Sep 4, 2020 at 6:39 AM Timo Walther wrote: > +1 for getting

[DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

Hi Devs, I'd like to propose an update to how state backends and checkpoint storage are configured to help users better understand Flink. Apache Flink's durability story is a mystery to many users. One of the most common recurring questions from users comes from not understanding the relationship

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

this, though. > > Aljoscha > > On 09.09.20 10:05, Konstantin Knauf wrote: > > Thanks for the initiative. Big +1. Would be interested to hear if the > > proposed interfaces still make sense in the face of the new > fault-tolerance > > work that is planned. St

Re: [DISCUSS] Deprecate and remove UnionList OperatorState

Generally +1 The one use case I've seen of union state I've seen in production (outside of sources and sinks) is as a "poor mans" broadcast state. This was obviously before that feature was added which is now a few years ago so I don't know if those pipelines still exist. FWIW, if they do the stat

Re: add support for ScalaPB-based message-payload serialization to Stateful Functions?

Hi Galen, Welcome to the dev list! I'll make sure the right person responds to the ticket. Seth On Wed, Sep 9, 2020 at 5:27 PM Galen Warren wrote: > Hi all -- I created a ticket > regarding a proposal to > add a new message-payload serializati

[DISCUSS] Drop Scala 2.11

Hi Everyone, Think of this as a pre-flip, but what does everyone think about dropping Scala 2.11 support from Flink. The last patch release was in 2017 and in that time the scala community has released 2.13 and is working towards a 3.0 release. Apache Kafka and Spark have both dropped 2.11 suppor

Re: [DISCUSS] Drop Scala 2.11

since it's blocking us from upgrading > > certain dependencies. > > > > I would also be in favour of dropping Scala completely but that's a > > different story. > > > > Aljoscha > > > > On 10.09.20 16:51, Seth Wiesman wrote: > >

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

Having thought about it more, HashMapStateBackend has won me over. I'll update the FLIP. If there aren't any more comments I'll open it up for voting on monday. Seth On Wed, Sep 9, 2020 at 9:09 AM Seth Wiesman wrote: > @Yun yes, this is really about making CheckpointSto

Re: [VOTE] FLIP-140: Introduce bounded style execution for keyed streams

+1 (binding) Seth On Thu, Sep 10, 2020 at 9:13 AM Aljoscha Krettek wrote: > +1 (binding) > > Aljoscha >

[VOTE] FLIP-142: Disentangle StateBackends from Checkpointing

Hi all, After the discussion in [1], I would like to open a voting thread for FLIP-142 [2] which discusses disentangling state backends from checkpointing. The vote will be open until 16th September (72h), unless there is an objection or not enough votes. Seth [1] http://apache-flink-mailing-li

Re: [ANNOUNCE] New Apache Flink Committer - Igal Shilman

Congrats Igal! On Tue, Sep 15, 2020 at 7:13 AM Benchao Li wrote: > Congratulations! > > Zhu Zhu 于2020年9月15日周二 下午6:51写道： > > > Congratulations, Igal! > > > > Thanks, > > Zhu > > > > Rafi Aroch 于2020年9月15日周二 下午6:43写道： > > > > > Congratulations Igal! Well deserved! > > > > > > Rafi > > > > > > >

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

ljoscha Krettek wrote: > > I could try and come up with a longer name if you need it ... ;-) > > > > Aljoscha > > > > On 11.09.20 16:25, Seth Wiesman wrote: > >> Having thought about it more, HashMapStateBackend has won me over. I'll > >>

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

should just do these changes. The impact should be minimal. > > Best, > > Dawid > > > On 15/09/2020 15:20, Seth Wiesman wrote: > > Hey Dawid, > > > > I didn't want to break compatibility but if there is precedent and > everyone > > is ok with it

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

t; 'state.backend.async' and > `state.backend.rocksdb.checkpoint.transfer.thred.num` will be configured > after the separation, I think these configurations are more related to > snapshots (maybe a little strange to configure these on statebackend side). > did not see this on the FLIP wiki currently. >

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

eckpoint storage involved, > > for example for savepoints or for backup/consolidation, so no problem. > > > > > > Best, > > Stephan > > > > On Wed, Sep 16, 2020 at 5:11 PM Aljoscha Krettek > > wrote: > > > >> I think the mentioned setting

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

sal of solution yet, but I suggest we figure > this out clearly. > > Thanks. > > Best Regards, > Yu > > > On Thu, 17 Sep 2020 at 13:19, Congxian Qiu wrote: > > > Thanks for the detailed replay, +1 from my side. > > Best, > > Congxian > > > >

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

"CheckpointStorageAccess" and then use the name "CheckpointStorage" instead > of "SnapshotStorage". > > > > On Thu, Sep 17, 2020 at 3:47 PM Seth Wiesman wrote: > > > Hi Yu, > > > > I've updated the Deprecation / Compatibility

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

lean them up > 2. Personally I'm in favor of `JobManagerCheckpointStorage` / > `FileSystemCheckpointStorage` than `JobManagerStorage` / > `FileSystemStorage` > > Thanks. > > Best Regards, > Yu > > > On Fri, 18 Sep 2020 at 01:58, Seth Wiesman wrote: > > > That

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

be honest, I find > this part ambiguous between JM and FS checkpoint storage) > > Please let me know your thoughts. Thanks. > > Best Regards, > Yu > > > On Fri, 18 Sep 2020 at 23:02, Seth Wiesman wrote: > > > 1. With `FSStateBackend`, we used to decide wher

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

t; > file system, to make it externally accessible/addressable, and for master > > failover (HA). > > - The file state size threshold is the threshold where data is stored > > inline with the metadata, rather than in a separate file. Whatever the > > JobManager does wit

Re: [VOTE] Apache Flink Stateful Functions 2.2.0, release candidate #2

+1 (non binding) - Verified signatures and checksums - Checked licenses and notices - Clean build from source - Executed all end to end tests - Deployed to K8s with Go SDK (no sdk changes required btw :)) - Simulated TM and remote function failure and verified recovery - Checked updated docs Seth

Re: [DISCUSS] Move Hive document to "Table & SQL Connectors" from "Table API & SQL"

+1 On Thu, Sep 24, 2020 at 2:49 AM Rui Li wrote: > +1 > > On Thu, Sep 24, 2020 at 2:59 PM Timo Walther wrote: > > > +1 > > > > On 24.09.20 06:54, Jark Wu wrote: > > > +1 to move it there. > > > > > > On Thu, 24 Sep 2020 at 12:16, Jingsong Li > > wrote: > > > > > >> Hi devs and users: > > >> >

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

Hi Yu, bq* I thought the FLIP aims at resolving some *existing* confusion, i.e. the durability mystery to users. I think it might help to highlight specific stumbling blocks users have today and why I believe this change addresses those issues. Some frequent things I've heard over the past severa

Re: [DISCUSS] FLIP-142: Disentangle StateBackends from Checkpointing

ough the current javadoc of `StateBackend` states > "MemoryStateBackend is not highly available", it actually supports > persisting the checkpoint data to DFS when checkpoint path is given, so the > mapping between old and new APIs are not that straight-forward and need > some

[RESULT][VOTE] FLIP-142: Disentangle StateBackends from Checkpointing

Hi all, The voting time for FLIP-142 has passed. I'm closing the vote now. - Konstantin (binding) - David Anderson (binding) - Gordon (binding) - Congxian - David Wysakowicz (binding) - Aljoscha (binding) - Yu (binding) - Kostas (binding) Including myself, there were 9 votes, 8 binding. There we

[Announce] Flink Forward Global Is Just Around the Corner

final Keynote for Ververica will be announced next week. As a reminder, the event is virtual and free to attend[1]. There are also a limited number of paid training slots available. Looking forward to seeing everyone virtually soon! https://www.flink-forward.org/ Seth Wiesman - Committer Apache Flink

Re: [DISCUSS] Remove flink-connector-filesystem module.

+1 It has been deprecated for some time and the StreamingFileSink has stabalized with a large number of formats and features. Plus, the bucketing sink only implements a small number of stable interfaces[1]. I would expect users to continue to use the bucketing sink from the 1.11 release with futur

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

of mixing them in one connector. The new connector > >>>>>> requires > >>>>>>>>> hash sink partitioner, primary key declared, regular format. > >>>>>>>>> If we mix them in one connector, it might be confusi

[reminder] Build and check the documentation when reviewing

Hi Devs, First of all, I'd like to thank everyone for being more proactive about writing documentation along with their features. Compared to previous releases there is not so much last-minute documentation writing and that is something to be celebrated! That said, we are still beginning to see a

[DISCUSS] Make SQL docs Blink only

Hi Everyone, I've been spending a lot of time recently working on the SQL documentation and I'm finding it very difficult to explain semantics as the two table planners continue to diverge. As Blink has been the default planner for some time, and 1.12 now offers bounded data stream support, how do

Re: [DISCUSS] Make SQL docs Blink only

wrote: > >>>>> > >>>>>> +1 > >>>>>> > >>>>>> Yes, please! > >>>>>> > >>>>>> On 08.12.20 16:52, David Anderson wrote: > >>>>>>> I agree -- I t

Re: [DISCUSS] Releasing StateFun hotfix version 2.2.2

+1 This is an important fix, and I'm available this week for RC testing. Given the makeup of the normal statefun contributors, it might make sense to proactively reach out to some additional PMC members about testing. Seth On Mon, Dec 21, 2020 at 4:18 AM Tzu-Li (Gordon) Tai wrote: > Hi, > > No

Re: [DISCUSS] FLIP-155: Introduce a few convenient operations in Table API

This makes sense, I have some questions about method names. What do you think about renaming `dropDuplicates` to `deduplicate`? I don't think that drop is the right word to use for this operation, it implies records are filtered where this operator actually issues updates and retractions. Also, de

[DISCUSS] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

Hi All, I would like to start a discussion for FLIP-157: Migrating the Flink docs from Jekyll to Hugo. This will allow us: - Proper internationalization - Working Search - Sub-second build time ;) Please take a look and let me know what you think. Seth https://cwiki.apache.org/conflu

Re: [DISCUSS] Backport broadcast operations in BATCH mode to Flink

+1 I would hope this helps attract more early adopters so if there are issues we can resolve them in time for 1.13. Seth On Wed, Jan 13, 2021 at 5:13 AM Dawid Wysakowicz wrote: > Hi, > > Given that the BATCH execution mode was only released in 1.12 and a > rather small impact of the suggested

Re: [DISCUSS] Deprecate MapR FS

+1 I actually thought we had already dropped this FS. If anyone is still relying on it in production, the file system abstraction in Flink has been incredibly stable over the years. They should be able to use the 1.14 MapR FS with later versions of Flink. Seth On Wed, Dec 8, 2021 at 10:03 AM Mar

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

+1 (non-binding) - Checked Log4J version and updated license preambles on all releases - Verified signatures on sources - Reviewed blog post Seth On Mon, Dec 13, 2021 at 1:42 PM Jing Ge wrote: > +1 LGTM. Many thanks for your effort! > > On Mon, Dec 13, 2021 at 8:28 PM Chesnay Schepler > wro

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

Thank you for managing these updates Chesnay! On Tue, Dec 14, 2021 at 3:51 PM Chesnay Schepler wrote: > Since the maven artifacts have already been published we will use the > next patch version for each release, i.e.: > 1.11.6 > 1.12.7 > 1.13.5 > 1.14.2 > > (We could technically just update t

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

+1 (non-binding) - Checked diff of all versions and verified dep upgrade - Verified checksum and signatures - Built 1.14 from source - checked blog post Seth On Wed, Dec 15, 2021 at 10:22 AM Yu Li wrote: > +1 > > * Verified checksums and signatures > * Reviewed website PR >- Minor: left a

Re: [DISCUSS] Immediate dedicated StateFun releases for log4j vulnerability

+1 On Thu, Dec 16, 2021 at 12:37 PM Igal Shilman wrote: > Hi All, > > Following the recent Apache Flink releases due to the log4j vulnerability, > I'd like to propose an immediate StateFun release - 3.1.1. > This release is basically the same as 3.1 but updates the Flink dependency > to 1.13.3.

Re: [DISCUSS] Immediate dedicated StateFun releases for log4j vulnerability

And I'm happy to help with the release. On Thu, Dec 16, 2021 at 12:55 PM Seth Wiesman wrote: > +1 > > On Thu, Dec 16, 2021 at 12:37 PM Igal Shilman wrote: > >> Hi All, >> >> Following the recent Apache Flink releases due to the log4j vulnerability, >>

Re: [jira] [Created] (FLINK-25197) Using Statefun RequestReplyFunctionBuilder fails with Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jackso

Hi Galen, Sorry for the late reply, a lot of people are on sporadic schedules with the holidays. A PR would be very welcome! I've gone ahead and assigned you the Jira. And I'll watch the repo to review your pr. cheers, Seth On Mon, Dec 20, 2021 at 8:24 AM Galen Warren wrote: > I just wanted t

Re: [VOTE] Stateful functions 3.1.1 release

+1 (non-binding) - Verified signatures - Checked diff - Checked site PR - Build from source and ran e2e tests Seth On Mon, Dec 20, 2021 at 10:59 AM Igal Shilman wrote: > Hi everyone, > > Please review and vote on the release candidate #2 for the version 3.1.1 of > Apache Flink Stateful Functio

Re: [DISCUSS] FLIP-203: Incremental savepoints

>> AFAIK state schema evolution should work both for native and canonical >> savepoints. Schema evolution does technically work for both formats, it happens after the code paths have been unified, but the community has up until this point considered that an unsupported feature. From my perspective

Re: [DISCUSS] Stateful Functions 3.2.0 release?

+1 These are already a useful set of features to release to our users. Seth On Mon, Jan 24, 2022 at 8:45 AM Till Rohrmann wrote: > Hi everyone, > > We would like to kick off a new StateFun release 3.2.0. The new release > will include the new JavaScript SDK and some useful major features: > >

Re: [VOTE] Apache Flink Stateful Functions 3.2.0, release candidate #1

+1 (non-binding) - verified checksums and signatures - built from source and ran e2e tests - checked dependencies / licenses - deployed greeter with golang sdk - reviewed blog post Cheers, Seth On Fri, Jan 28, 2022 at 4:22 AM Igal Shilman wrote: > +1 (binding) > > - verified checksum and sig

Re: Need help with finding inner workings of watermark stream idleness

Hi Jeff, I think the class you're looking for is StatusWatermarkValve. Note that this is fairly deep into the runtime stack. Seth On Tue, Feb 1, 2022 at 2:34 PM Jeff Carter wrote: > Thanks, Till. > > That definitely helps a bit. I'm still not seeing where there is some idle > variable that the

Re: [VOTE] Deprecate NiFi connector

+1 (binding) On Thu, Feb 3, 2022 at 8:44 AM Fabian Paul wrote: > Thanks for driving the deprecation efforts. > > +1 (binding) > > Best, > Fabian > > On Mon, Jan 31, 2022 at 11:47 AM Martijn Visser > wrote: > > > > Hi everyone, > > > > I would like to open up a vote to deprecate NiFi in Flink 1.

Re: [VOTE] Remove Twitter connector

+1 (binding) On Thu, Feb 3, 2022 at 8:39 AM Fabian Paul wrote: > This connector is really a relict of the past. > > +1 (binding) > > Best, > Fabian > > On Mon, Jan 31, 2022 at 11:47 AM Martijn Visser > wrote: > > > > Hi everyone, > > > > I would like to open up a vote to remove the Twitter conn

Re: question about StatefunContext in golang Statefun SDK

Hi Galen, No, that is not currently supported, the current idiomatic way would be to pass those values to the struct implementing the Statefun interface. type MyFunc struct { someRuntimeInfo string } func (m *MyFunc) Invoke(ctx statefun.Context, message statefun.Message) error { } func main() {

Re: question about StatefunContext in golang Statefun SDK

; > > >>> >> > > > The former. > > > >>> >> > > > > > > >>> >> > > > I think there's potential for confusion here because we're > > > >>> using the > > > >>>

Re: question about StatefunContext in golang Statefun SDK

> Hi Galen, > > Thanks for explaining the problems with the current design. I think I've > already learned quite a bit wrt Go thanks to you :-) > > From what you describe it seems indeed a bit restrictive to let the > statefun.Context contain the context.Context w/o giving access

Re: [VOTE] Release 1.14.4, release candidate #1

+1 non-binding - built from source - checked hashes and signatures - started locally and deployed wordcount / stopped with savepoint / restarted - reviewed announcement post Thanks for managing the release! Seth On Fri, Feb 25, 2022 at 7:30 AM Konstantin Knauf wrote: > Hi everyone, > > Please

Re: [DISCUSS] CAST legacy behaviour

+1 Especially as SQL upgrades are right around the corner, it makes sense to get our defaults right. Seth On Mon, Feb 28, 2022 at 7:14 AM Martijn Visser wrote: > +1 for setting this option to disabled by default. I believe failures > should be brought forward as soon as possible, so they can b

Re: [DISCUSS] Enable scala formatting check

Happy to help get this merged. Do we want to wait until the 1.15 branch is cut? The change is mostly trivial (reformatting) but does make changes to the build system. Seth On Wed, Mar 9, 2022 at 9:45 AM Francesco Guardiani wrote: > Hi all, > I've been spending some time prototyping a scalafmt

Re: [DISCUSS] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

> > 在 2021年1月14日，18:21，Aljoscha Krettek 写道： > > > > > > > > +1 > > > > > > > > The build times on Jekyll have just become to annoying for me. I > > realize that that is also a function of how we structure our > documentation, > >

Re: [DISCUSS] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

Great, if there aren't any other concerns I will open this up for a vote on Monday. Seth On Thu, Jan 14, 2021 at 9:03 AM Seth Wiesman wrote: > Happy to see there is enthusiasm for this change, let me try and answers > each of these questions. > > @Jark Wu Hugo has proper

  1   2   3   4   >