Re: [VOTE] Release 1.8.1, release candidate #1

+1 (binding) - I checked the diff in the POM files since 1.8.0 and they look good, i.e. no new dependencies that could lead to licensing problems > On 1. Jul 2019, at 10:02, Tzu-Li (Gordon) Tai wrote: > > +1 (binding) > > - checked signatures and hashes > - built from source without skipping

Re: [VOTE] Migrate to sponsored Travis account

+1 Aljoscha > On 4. Jul 2019, at 11:09, Stephan Ewen wrote: > > +1 to move to a private Travis account. > > I can confirm that Ververica will sponsor a Travis CI plan that is > equivalent or a bit higher than the previous ASF quota (10 concurrent build > queues) > > Best, > Stephan > > On Th

Re: [VOTE] How to Deal with Split/Select in DataStream API

I think this would benefit from a FLIP, that neatly sums up the options, and which then gives us also a point where we can vote and ratify a decision. As a gut feeling, I most like Option 3). Initially I would have preferred option 1) (because of a sense of API purity), but by now I think it’s g

Re: [DISCUSS] META-FLIP: Sticking (or not) to a strict FLIP voting process

think Jark raised a valid point. We should have a clear >>>>>>> understanding what binding votes in this case mean. I think it >>> makes >>>>>> sense >>>>>>> to consider PMC's and committers' votes as binding for FLI

Re: [DISCUSS] Flink project bylaws

Big +1 How different is this from the Kafka bylaws? I’m asking because I quite like them and wouldn’t mind essentially adopting the Kafka bylaws. I mean, it’s open source, and we don’t have to try to re-invent the wheel here. I think it’s worthwhile to discuss the “committer +1” requirement. We

Re: [DISCUSS] Flink project bylaws

>>> general definition of Apache glossary and other projects. >>>> 3. review board -> pull request >>>> >>>> - >>>> Re: Chesnay >>>> >>>> The emeritus stuff seems like unnecessary noise.

Re: [FLIP-47] Savepoints vs Checkpoints

Hi, Sorry for the quite late response! I initially understood FLIP-45 [0] more as a “allow user to stop-with-checkpoint”, that’s why I didn’t think too much about the other things it mentions like semantics of savepoints and checkpoints. I thought that the “stop-with-checkpoint” would work ver

Re: subscribe flink dev mail list

Hi, To subscribe to the dev mailing list you have to send an email to dev-subscr...@flink.apache.org Aljoscha > On 18. Jul 2019, at 10:58, venn wrote: > > I'am glad to subscribe the dev mail list >

Re: [DISCUSS] Support temporary tables in SQL API

I would be fine with option 3) but I think option 2) is the more implicit solution that has less surprising behaviour. Aljoscha > On 22. Jul 2019, at 23:59, Xuefu Zhang wrote: > > Thanks to Dawid for initiating the discussion. Overall, I agree with Timo > that for 1.9 we should have some quick

Re: [DISCUSS] Support temporary tables in SQL API

lso try to >> reuse the in-memory catalog >> for the builtin temporary table storage. >> >> Regarding to option #2 and option #3, from user's perspective, IIUC option >> #2 allows user to have >> simple name to reference temporary table and should use fully qua

Re: [DISCUSS] Support temporary tables in SQL API

opriate PRs according to the decision (unless somebody > objects). We will revisit the long-term solution in a separate thread as part > of the 1.10 release after 1.9 is released. > > Thank you all for your opinions! > > Best, > > Dawid > > On 24/07/2019 09:35, A

Re: flink-mapr-fs failed in travis

I’m seeing the same issue when building this locally. I’ll start a DISCUSS thread so see about just removing the flink-mapr-fs module. Aljoscha > On 24. Jul 2019, at 05:00, JingsongLee > wrote: > > Sorry, > "It looks like it's been compiled all the time." > should be: "It looks like it can

[DISCUSS] Removing the flink-mapr-fs module

Hi, Because of recent problems in the dependencies of that module [1] I would suggest that we remove it. If people are using it, they can use the one from Flink 1.8. What do you think about it? It would a) solve the dependency problem and b) make our build a tiny smidgen more lightweight. Alj

Re: [DISCUSS] Removing the flink-mapr-fs module

l 29, 2019 at 5:19 PM Ufuk Celebi wrote: > >> +1 >> >> >> On Mon, Jul 29, 2019 at 11:06 AM Jeff Zhang wrote: >> >>> +1 to remove it. >>> >>> Aljoscha Krettek 于2019年7月29日周一 下午5:01写道： >>> >>>> Hi, >>>>

Re: [DISCUSS] Removing the flink-mapr-fs module

-- > From:Biao Liu > Send Time:2019年7月29日(星期一) 12:05 > To:dev > Subject:Re: [DISCUSS] Removing the flink-mapr-fs module > > +1 for removing it. > > Actually I encountered this issue several times. I thought it might be > blocked by firewall of China :( >

Re: [DISCUSS][CODE STYLE] Usage of Java Optional

Hi, I’m also in favour of using Optional only for method return values. I could be persuaded to allow them for parameters of internal methods but I’m sceptical about that. Aljoscha > On 2. Aug 2019, at 15:35, Yu Li wrote: > > TL; DR: I second Timo that we should use Optional only as method r

Re: [VOTE] Publish the PyFlink into PyPI

+1 (binding) > On 1. Aug 2019, at 17:23, Hequn Cheng wrote: > > +1 (non-binding) > > Thanks a lot for driving this! @jincheng sun > > Best, Hequn > > On Thu, Aug 1, 2019 at 11:00 PM Biao Liu wrote: > >> Thanks Jincheng for working on this. >> >> +1 (non-binding) >> >> Thanks, >> Biao /'b

Re: [VOTE] Flink Project Bylaws

+1 > On 11. Aug 2019, at 10:07, Becket Qin wrote: > > Hi all, > > I would like to start a voting thread on the project bylaws of Flink. It > aims to help the community coordinate more smoothly. Please see the bylaws > wiki page below for details. > > https://cwiki.apache.org/confluence/pages/v

Re: [DISCUSS] Drop stale class Program

Hi, I would be in favour of removing Program (and the code paths that support it) for Flink 1.10. Most users of Flink don’t actually know it exists and it is only making our code more complicated. Going forward with the new Client API discussions will be a lot easier without it as well. Best,

Re: [VOTE] Apache Flink Release 1.9.0, release candidate #2

+1 I did some testing on a Google Cloud Dataproc cluster (it gives you a managed YARN and Google Cloud Storage (GCS)): - tried both YARN session mode and YARN per-job mode, also using bin/flink list/cancel/etc. against a YARN session cluster - ran examples that write to GCS, both with the na

Re: [DISCUSS] FLIP-52: Remove legacy Program interface.

+1 (for the same reasons I posted on the other thread) > On 14. Aug 2019, at 15:03, Zili Chen wrote: > > +1 > > It could be regarded as part of Flink client api refactor. > Removal of stale code paths helps reason refactor. > > There is one thing worth attention that in this thread[1] Thomas >

Re: [VOTE] FLIP-51: Rework of the Expression Design

+1 This seems to be a good refactoring/cleanup step to me! > On 16. Aug 2019, at 10:59, Dawid Wysakowicz wrote: > > +1 from my side > > Best, > > Dawid > > On 16/08/2019 10:31, Jark Wu wrote: >> +1 from my side. >> >> Thanks Jingsong for driving this. >> >> Best, >> Jark >> >> On Thu, 15

Re: [DISCUSS] Reducing build times

Speaking of flink-shaded, do we have any idea what the impact of shading is on the build time? We could get rid of shading completely in the Flink main repository by moving everything that we shade to flink-shaded. Aljoscha > On 16. Aug 2019, at 14:58, Bowen Li wrote: > > +1 to Till's points

Re: [DISCUSS] Flink client api enhancement for downstream project

;>>> >>>>>>>>> 1) CliFrontend should exactly be a front end, at least for >>> "run" >>>>>>> command. >>>>>>>>> That means it just gathered and passed all config from >> command >>>> line >>&

Re: [DISCUSS] Reducing build times

any tests, API >> compatibility checks, checkstyle, layered shading (e.g., flink-runtime >> and flink-dist, where both relocate dependencies and one is bundled by >> the other), and, most importantly, CI (and really, without CI being >> covered in a PoC there's nothing

Re: [VOTE] Apache Flink 1.9.0, release candidate #3

+1 I checked the last RC on a GCE cluster and was satisfied with the testing. The cherry-picked commits didn’t change anything related, so I’m forwarding my vote from there. Aljoscha > On 21. Aug 2019, at 13:34, Chesnay Schepler wrote: > > +1 (binding) > > On 21/08/2019 08:09, Bowen Li wrot

Re: [VOTE] FLIP-52: Remove legacy Program interface.

+1 > On 21. Aug 2019, at 13:30, Chesnay Schepler wrote: > > +1 > > On 21/08/2019 13:23, Timo Walther wrote: >> +1 >> >> Am 21.08.19 um 13:21 schrieb Stephan Ewen: >>> +1 >>> >>> On Wed, Aug 21, 2019 at 1:07 PM Kostas Kloudas wrote: >>> Hi all, Following the FLIP process, thi

Re: [VOTE] Release flink-shaded 8.0, release candidate #1

+1 (binding) - I verified the signature and checksum - I eyeballed the list of resolved issues - I checked the maven central artifices Aljoscha > On 23. Aug 2019, at 21:05, Chesnay Schepler wrote: > > Hi everyone, > Please review and vote on the release candidate #1 for the version 8.0, as

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

Overall, this is a very nice development that should also simplify the code base once we deprecate the expression parser! Regarding method names, I agree with Seth that values/literals should use something like “lit()”. I also think that for column references we could use “col()” to make it cle

Re: ClassLoader created by BlobLibraryCacheManager is not using context classloader

Hi, I actually don’t know whether that change would be ok. FlinkUserCodeClassLoader has taken FlinkUserCodeClassLoader.class.getClassLoader() as the parent ClassLoader before my change. See: https://github.com/apache/flink/blob/release-1.2/flink-runtime/src/main/java/org/apache/flink/runtime/ex

Re: [DISCUSS] Releasing Flink 1.8.2

I cut a PR for FLINK-13586: https://github.com/apache/flink/pull/9595 > On 2. Sep 2019, at 05:03, Yu Li wrote: > > +1 for a 1.8.2 release, thanks for bringing this up Jincheng! > > Best Regards, > Yu > > > On Mon, 2 Sep 2019 at 09:19, Thomas Weise

Re: ClassLoader created by BlobLibraryCacheManager is not using context classloader

loader when > using LocalEnvironment, but that looks a little odd. > > Jan > > On 9/2/19 11:02 AM, Aljoscha Krettek wrote: >> Hi, >> >> I actually don’t know whether that change would be ok. >> FlinkUserCodeClassLoader has taken >> FlinkUserCodeClassLo

Re: [DISCUSS] FLIP-54: Evolve ConfigOption and Configuration

Hi, Regarding the factory and duplicate() and whatnot, wouldn’t it work to have a factory like this: /** * Allows to read and write an instance from and to {@link Configuration}. A configurable instance * operates in its own key space in {@link Configuration} and will be (de)prefixed by the

Re: [DISCUSS] FLIP-54: Evolve ConfigOption and Configuration

Hi, I think it’s important to keep in mind the original goals of this FLIP and not let the scope grow indefinitely. As I recall it, the goals are: - Extend the ConfigOption system enough to allow the Table API to configure options that are right now only available on CheckpointingOptions, Exe

Re: [DISCUSS] Releasing Flink 1.8.2

Hi, I’m just running the last tests on FLINK-13586 on Travis and them I’m merging. Best, Aljoscha > On 4. Sep 2019, at 07:37, Jark Wu wrote: > > Thanks for the work Jincheng! > > I have moved remaining major issues to 1.8.3 except FLINK-13586. > > Hi @Aljoscha

Re: [DISCUSS] Flink Python User-Defined Function for Table API

>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>> https://cwiki.ap

Re: [DISCUSS] Flink Python User-Defined Function for Table API

hink you are right, we should try > to ensure the changes of this feature is minimal. For more detail we would > follow this principle when review the PRs. > > Great thanks for your questions and remind! > > Best, > Jincheng > > > Aljoscha Krettek 于2019年9月4日周三

Re: [DISCUSS] Flink Python User-Defined Function for Table API

an call > Catalog.create_function. There will be no API change per my understanding. > > What do you think? > Best,Jincheng > > Aljoscha Krettek 于2019年9月5日周四 下午5:32写道： > >> Hi, >> >> Another thing to consider is the Scope of the FLIP. Currently, we try to

Re: [DISCUSS] Flink Python User-Defined Function for Table API

> >>> Sure, for ensure the 1.10 relesae of flink, let's split the FLIPs, and >>> FLIP-58 only do the stateless part. >>> >>> Cheers, >>> Jincheng >>> >>> Aljoscha Krettek 于2019年9月6日周五 下午5:53写道： >>> >>>>

Re: Call for approving Elasticsearch 7.x connector

Hi, Thanks for the heads up! I commented on the issue. Best, Aljoscha > On 9. Sep 2019, at 10:38, vino yang wrote: > > Hi guys, > > There is an issue about supporting Elasticsearch 7.x.[1] > Based on our validation and discussion. We found that Elasticsearch 7.x > does not guarantee API compa

Re: [DISCUSS] Flink client api enhancement for downstream project

ether a dedicate thread or a Slack group is the proper one. In my opinion > we can > involve the team in a Slack group, concurrent with the FLIP process start > our branch > and once we reach a consensus on the FLIP, open an umbrella issue about the > framework > and start subtasks.

Re: [DISCUSS] Features for Apache Flink 1.10

Hi, Thanks for putting together the list! And I’m +1 for the suggested release timeline and also for Gary and Yu as the release managers. Best, Aljoscha On 9 Sep 2019, at 7:39, Yu Li wrote: Hi Xuefu, If I understand it correctly, the data type support work should be included in the "Table

Re: [PROPOSAL] Merge NewClusterClient into ClusterClient

I agree that NewClusterClient and ClusterClient can be merged now that there is no pre-FLIP-6 code base anymore. Side note, there are a lot of methods in ClusterClient that should not really be there, in my opinion: - all the getOptimizedPlan*() method - the run() methods. In the end, only sub

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

Hi, I think this discussion and the one for FLIP-64 are very connected. To resolve the differences, think we have to think about the basic principles and find consensus there. The basic questions I see are: - Do we want to support overriding builtin functions? - Do we want to support overridi

Re: [DISCUSS] Does removing deprecated interfaces needs another FLIP

I would say a ML discussion or even a Jira issue is enough because a) the methods are already deprecated b) the methods are @PublicEvolving, which I don't consider a super strong guarantee to users (we still shouldn't remove them lightly, but we can if we have to...) Best, Aljoscha On 07.02.

Re: [Discussion] Job generation / submission hooks & Atlas integration

If we need it, we can probably beef up the JobListener to allow accessing some information about the whole graph or sources and sinks. My only concern right now is that we don't have a stable interface for our job graphs/pipelines right now. Best, Aljoscha On 06.02.20 23:00, Gyula Fóra wrote:

Re: [DISCUSS] Drop connectors for Elasticsearch 2.x and 5.x

+1 for dropping them, this stuff is quite old by now. On 10.02.20 15:04, Benchao Li wrote: +1 for dropping 2.x - 5.x. FYI currently only 6.x and 7.x ES Connectors are supported by table api. Flavio Pompermaier 于2020年2月10日周一 下午10:03写道： +1 for dropping all Elasticsearch connectors < 6.x On M

Re: [VOTE] FLIP-55: Introduction of a Table API Java Expression DSL

+1 Best, Aljoscha On 11.02.20 11:17, Jingsong Li wrote: Thanks Dawid for your explanation, +1 for vote. So I am big +1 to accepting java.lang.Object in the Java DSL, without scala implicit conversion, a lot of "lit" look unfriendly to users. Best, Jingsong Lee On Tue, Feb 11, 2020 at 6:07 P

Re: [Discussion] Job generation / submission hooks & Atlas integration

gs. The Atlas hook works on this map. This is very fragile and depends on a lot of internals. Kind of like exposing the JobGraph but much worse. I think we can do better. Gyula On Fri, Feb 7, 2020 at 9:55 AM Aljoscha Krettek wrote: If we need it, we can probably beef up the JobListener to al

Re: [DISCUSS] Improve history server with log support

Hi, what's the difference in approach to the mentioned related Jira Issue ([1])? I commented there because I'm skeptical about adding Hadoop-specific code to the generic cluster components. Best, Aljoscha [1] https://issues.apache.org/jira/browse/FLINK-14317 On 13.02.20 03:47, SHI Xiaogang

[ANNOUNCE] New Documentation Style Guide

Hi Everyone, we just merged a new style guide for documentation writing: https://flink.apache.org/contributing/docs-style.html. Anyone who is writing documentation or is planning to do so should check this out. Please open a Jira Issue or respond here if you have any comments or questions.

Re: [ANNOUNCE] New Documentation Style Guide

Wu 写道： Great summary! Thanks for adding the translation specification in it. I learned a lot from the guide. Best, Jark On Fri, 14 Feb 2020 at 23:39, Aljoscha Krettek < aljos...@apache.org> wrote: Hi Everyone, we just merged a new style guide for documentation writing: https://flin

Re: [DISCUSS] Drop connectors for Elasticsearch 2.x and 5.x

Wouldn't removing the ES 2.x connector be enough because we can then update the ES 5.x connector? It seems there are some users that still want to use that one. Best, Aljoscha On 18.02.20 10:42, Robert Metzger wrote: The ES5 connector is causing some problems on the CI system. It would be nic

Re: [DISCUSS] Improvements on FLIP Process

Hi, thanks for starting this discussion! However, I have a somewhat opposing opinion to this: we should disallow using Google Docs for FLIPs and FLIP discussions and follow the already established process more strictly. My reasons for this are: - discussions on the Google Doc are not reflec

Re: Hotfixes on the master

ld expect. Greg Note: this summary is rather naive and includes non-squashed hotfix commits included in a PR $ git shortlog --grep 'hotfix' -s -n release-0.9.0.. 94 Stephan Ewen 42 Aljoscha Krettek 20 Till Rohrmann 16 Robert Metzger 13 Ufuk Celebi

Re: [DISCUSS] Kicking off the 1.11 release cycle

+1 Although I would hope that it can be more than just "anticipated". Best, Aljoscha On 19.02.20 15:40, Till Rohrmann wrote: Thanks for volunteering as one of our release managers Zhijiang. +1 for the *anticipated feature freeze date* end of April. As we go along and collect more data points

[DISCUSS] Extend (or maintain) "shell" script support for Windows

Hi, the background is this series of Jira Issues and PRs around extending the .bat scripts for windows: https://issues.apache.org/jira/browse/FLINK-5333. I would like to resolve this, by either closing the Jira Issues as "Won't Do" or finally merging these PRs. The questions I have are: -

Re: [PROPOSAL] Reverse the dependency from flink-streaming-java to flink-client

cidentally, ProgramInvocationException, we just throw in place as it is accessible. - transitively, flink-optimizer, for one utility. - transitively, flink-java, for several utilities. flink-runtime: - mainly for JobGraph generating. With a previous discussion with @Aljoscha Krettek our goal is brie

Re: [DISCUSS] Disable "Squash and merge" button for Flink repository on GitHub

If there is a noreply email address that could be on purpose. This happens when you configure github to not show your real e-mail address. This also happens when contributors open a PR and don't want to show their real e-mail address. I talked to at least 1 person that had it set up like this o

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

> For the -R flag, this was in the PoC that I published just as a quick > implementation, so that I can move fast to the entrypoint part. > Personally, I would not even be against having a separate command in > the CLI for this, sth like run-on-cluster or something along those > lines. > What do y

Re: [PROPOSAL] Reverse the dependency from flink-streaming-java to flink-client

On 09.03.20 03:15, tison wrote: So far, there is a PR[1] that implements the proposal in this thread. I look forward to your reviews or start a vote if required. Nice, I'll try and get to review that this week. Best, Aljoscha

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

Since there was no-one that said we should keep the windows scripts and no-one responded on the user ML thread I'll close the Jira issues/PRs about extending the scripts. Aljoscha

Re: [DISCUSS] FLIP-85: Delayed Job Graph Generation

On 10.03.20 03:31, Yang Wang wrote: For the "run-job", do you mean to submit a Flink job to an existing session or just like the current per-job to start a dedicated Flink cluster? Then will "flink run" be deprecated? I was talking about the per-job mode that starts a dedicated Flink cluster.

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

On 10.03.20 14:35, Robert Metzger wrote: I'm wondering whether we should file a ticket to remove the *.bat files in bin/ ? We can leave them there because they're not doing much harm, and removing them might actively break some existing setup. Best, Aljoscha

Re: [Discussion] Job generation / submission hooks & Atlas integration

Thanks! I'm reading the document now and will get back to you. Best, Aljoscha

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

Hi, I don't understand this discussion. Hints, as I understand them, should work like this: - hints are *optional* advice for the optimizer to try and help it to find a good execution strategy - hints should not change query semantics, i.e. they should not change connector properties executi

Re: Flink Kafka consumer auto-commit timeout

On 09.03.20 06:10, Rong Rong wrote: - Is this feature (disabling checkpoint and restarting job from Kafka committed GROUP_OFFSET) not supported? I believe the Flink community never put much (any?) effort into this because the Flink Kafka Consumer does its own offset handling. Starting from th

Re: [VOTE] [FLIP-85] Flink Application Mode

+1 (binding) Aljoscha

Re: [VOTE] [FLIP-85] Flink Application Mode

On 12.03.20 14:05, Flavio Pompermaier wrote: There's also a related issue that I opened a long time ago https://issues.apache.org/jira/browse/FLINK-10879 that could be closed once implemented this FLIP (or closed immediately and referenced as duplicated by the new JIRA ticket that would be creat

Re: [DISCUSS] Drop Bucketing Sink

+1 Aljoscha

Re: Flink Kafka consumer auto-commit timeout

Thanks for the update! On 13.03.20 13:47, Rong Rong wrote: 1. I think we have finally pinpointed what the root cause to this issue is: When partitions are assigned manually (e.g. with assign() API instead subscribe() API) the client will not try to rediscover the coordinator if it dies [1]. This

Re: PackagedProgram and ProgramDescription

On 18.03.20 14:45, Flavio Pompermaier wrote: what do you think if we exploit this job-submission sprint to address also the problem discussed in https://issues.apache.org/jira/browse/FLINK-10862? That's a good idea! What should we do? It seems that most committers on the issue were in favour o

Re: [DISCUSS] FLIP-84 Feedback Summary

Agreed to what Dawid and Timo said. To answer your question about multi line SQL: no, we don't think we need this in Flink 1.11, we only wanted to make sure that the interfaces that we now put in place will potentially allow this in the future. Best, Aljoscha On 01.04.20 09:31, godfrey he wr

Re: [DISCUSS] Change default planner to blink planner in 1.11

+1 to making Blink the default planner, we definitely don't want to maintain two planners for much longer. Best, Aljoscha

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

I think we're designing ourselves into ever more complicated corners here. Maybe we need to take a step back and reconsider. What would you think about this (somewhat) simpler proposal: - introduce a hint called CONNECTOR_OPTIONS(k=v,...). or CONNECTOR_PROPERTIES, depending on what naming we w

[PSA] Please report all occurrences of test failures

Hi All, we're currently struggling a bit with test stability, it seems especially on Azure. If you encounter a test failure in a PR or anywhere else, please take the time to check if there is already a Jira issue or create a new one. If there is already an Issue, please report the additional

Re: [VOTE] FLIP-110: Support LIKE clause in CREATE TABLE

+1 Aljoscha

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

at do you think? Regards, Timo On 02.04.20 16:24, Aljoscha Krettek wrote: I think we're designing ourselves into ever more complicated corners here. Maybe we need to take a step back and reconsider. What would you think about this (somewhat) simpler proposal: - introduce a hint called C

Re: [DISCUSS] FLIP-124: Add open/close and Collector to (De)SerializationSchema

On 07.04.20 08:45, Dawid Wysakowicz wrote: @Jark I was aware of the implementation of SinkFunction, but it was a conscious choice to not do it that way. Personally I am against giving a default implementation to both the new and old methods. This results in an interface that by default does not

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

On 08.04.20 04:27, Jark Wu wrote: I have a minor concern about the global configuration `table.optimizer.dynamic-table-options.enabled`, does it belong to optimizer? From my point of view, it is just an API to set table options and uses Calcite in the implementation. I'm also thinking about wha

[PSA] Please check your github email configuration when merging on Github

Hi Everyone, we have a lot of commits recently that were committed by "GitHub ". This happens when your GitHub account is not configured correctly with respect to your email address. Please make sure that your commits somehow show who is the committer. For reference, check out https://help.

Re: Configuring autolinks to Flink JIRA ticket in github repos

That is very nice! Thanks for taking care of this ~3q On 09.04.20 11:08, Dian Fu wrote: Cool! Thanks Yun for this effort. Very useful feature. Regards, Dian 在 2020年4月9日，下午4:32，Yu Li 写道： Great! Thanks for the efforts Yun. Best Regards, Yu On Thu, 9 Apr 2020 at 16:15, Jark Wu wrote: Tha

Re: [DISCUSS] FLIP-124: Add open/close and Collector to (De)SerializationSchema

On 10.04.20 17:35, Jark Wu wrote: 1) For correctness, it is necessary to perform the watermark generation as early as possible in order to be close to the actual data generation within a source's data partition. This is also the purpose of per-partition watermark and event-time alignment. Man

[DISCUSS] Releasing "fat" and "slim" Flink distributions

Hi Everyone, I'd like to discuss about releasing a more full-featured Flink distribution. The motivation is that there is friction for SQL/Table API users that want to use Table connectors which are not there in the current Flink Distribution. For these users the workflow is currently roughly

Re: [VOTE] FLIP-108: Add GPU support in Flink

Is the only really new method on the public APIs getExternalResourceInfos(..) on the RuntimeContext? I'm generally quite skeptical about adding anything to that interface but the method seems ok. Side note for the configuration keys: the pattern is similar to metrics configuration. There we ha

Re: [DISCUSS] Integration of training materials into Apache Flink

I'd be very happy if someone took over that part of the documentation! There are open issues for the TODOs in the concepts section here: https://issues.apache.org/jira/browse/FLINK-12639. But feel free to comment there/close/re-arrange as you see fit. Maybe we use this thread and Jira to coordi

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

ally verbose when I prepare webinars. At least, I think the flink-csv and flink-json should in the distribution, they are quite small and don't have other dependencies. Best, Jark On Wed, 15 Apr 2020 at 15:44, Jeff Zhang wrote: Hi Aljoscha, Big +1 for the fat flink distribution, where

Re: [VOTE] FLIP-124: Add open/close and Collector to (De)SerializationSchema

+1 (binding) Aljoscha

Re: [PROPOSAL] Google Season of Docs 2020.

Hi, first, excellent that you're driving this, Marta! By now I have made quite some progress on the FLIP-42 restructuring so that is not a good effort for someone to join now. Plus there is also [1], which is about incorporating the existing Flink Training material into the concepts section o

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

ot be chosen at the same time with kafka-universal etc. The benefit of it would be that the distribution that we release could remain "slim" or we could even make it slimmer. I might be missing something here though. Best, Dawdi On 16/04/2020 11:02, Aljoscha Krettek wrote: I want to

[DISCUSS] FLIP-126: Unify (and separate) Watermark Assigners

Hi Everyone! We would like to start a discussion on "FLIP-126: Unify (and separate) Watermark Assigners" [1]. This work was started by Stephan in an experimental branch. I expanded on that work to provide a PoC for the changes proposed in this FLIP: [2]. Currently, we have two different flav

Re: [DISCUSS] Adding support for Hadoop 3 and removing flink-shaded-hadoop

+1 to getting rid of flink-shaded-hadoop. But we need to document how people can now get a Flink dist that works with Hadoop. Currently, when you download the single shaded jar you immediately get support for submitting to YARN via bin/flink run. Aljoscha On 22.04.20 09:08, Till Rohrmann wro

Re: [DISCUSS] Exact feature freeze date

+1 Aljoscha On 23.04.20 15:23, Till Rohrmann wrote: +1 for extending the feature freeze until May 15th. Cheers, Till On Thu, Apr 23, 2020 at 1:00 PM Piotr Nowojski wrote: Hi Stephan, As release manager I’ve seen that quite a bit of features could use of the extra couple of weeks. This als

Re: [DISCUSS] Removing deprecated state methods in 1.11

Definitely +1! I'm always game for decreasing the API surface if it doesn't decrease functionality. Aljoscha On 23.04.20 14:18, DONG, Weike wrote: Hi Stephan, +1 for the removal, as there are so many deprecated methods scattered around, making APIs a little bit messy and confusing. Best, Wei

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

more. (2) I can imagine there still being a desire to have a "minimal" docker file, for users that want to keep the container images as small as possible, to speed up deployment. It is fine if that would not be the default, though. On Fri, Apr 17, 2020 at 12:16 PM Aljoscha Krettek wr

Re: Multiple rebalances are incorrectly ignored in some cases.

On 27.04.20 09:34, David Morávek wrote: When we include `flatMap` in between rebalances -> `.rebalance().flatMap(...).rebalance()`, we need to reshuffle again, because dataset distribution may have changed (eg. you can possibli emit unbouded stream from a single element). Unfortunatelly `flatMap

Re: A query on codebase exploration

Hi Manish, welcome to the community! You could start from a user program example and then try and figure out how that leads to job execution. So probably start with the DataStream WordCount example, figure out what the methods on DataStream do, that is how they build up a graph of Transformati

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

Hi, I’m fine with either signature for the new execute() method but I think we should focus on the executor discovery and executor configuration part in this FLIP while FLIP-74 is about the evolution of the method signature to return a future. I understand that it’s a bit weird, that this FLIP

Re: [DISCUSS] FLIP-74: Flink JobClient API

Hi Tison, Thanks for proposing the document! I had some comments on the document. I think the only complex thing that we still need to figure out is how to get a JobClient for a job that is already running. As you mentioned in the document. Currently I’m thinking that its ok to add a method to

Re: REST API / JarRunHandler: More flexibility for launching jobs

Hi, Regarding the original proposal: I don’t think spawning another process inside the JarHandler.runJar() is the way to go here. Looking at the bigger picture, the proposal would get us to roughly this situation: 1. Spawn Kubernetes containers (JobManager and TaskManagers) 2. User does a REST

  1   2   3   4   5   6   7   8   9   10   >