Re: [ANNOUNCE] Apache Flink 1.7.0 released

A great release, Thanks a lot for the great work Till! And also thanks to the whole Flink community! Let's moving forward. Best, Hequn On Sat, Dec 1, 2018 at 12:31 AM Tzu-Li Chen wrote: > Thanks a lot for being the release manager Till! And > thanks for the great work Flink community! > > Best

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Piotrek, Cache() should not affect semantics and business logic, and thus it will not lead to random behavior/results. The underlying design should ensure this. I thought your example as a valid anti-case. But Jiangjie is correct, the source table in batching should be immutable. It is the user

[jira] [Created] (FLINK-11044) RegisterTableSink docs incorrect

Kenny Gorman created FLINK-11044: Summary: RegisterTableSink docs incorrect Key: FLINK-11044 URL: https://issues.apache.org/jira/browse/FLINK-11044 Project: Flink Issue Type: Bug Co

Re: [DISCUSS] FLIP-27: Refactor Source Interface

Thanks everyone for the lively discussion. Let me try to summarize where I see convergence in the discussion and open issues. I'll try to group this by design aspect of the source. Please let me know if I got things wrong or missed something crucial here. For issues 1-3, if the below reflects the

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi all, Regarding naming `cache()` vs `materialize()`. One more explanation why I think `materialize()` is more natural to me is that I think of all “Table”s in Table-API as views. They behave the same way as SQL views, the only difference for me is that their live scope is short - current sess

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi all, I agree with @Becket that `cache()` and `materialize()` should be considered as two different methods where the later one is more sophisticated. According to my understanding, the initial idea is just to introduce a simple cache or persist mechanism, but as the TableAPI is a high-level

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Fabian, Thanks for sharing the feedback! Re: 1) Good question about the implementation. In fact, Alibaba has modified the query planning a little bit to add something called LogicalNodeBlock. Basically, a given DAG could be divided into a few LogicalNodeBlocks, and the optimization will be don

Re: [ANNOUNCE] Apache Flink 1.7.0 released

Thanks a lot for being the release manager Till! And thanks for the great work Flink community! Best, tison. vino yang 于2018年11月30日周五 下午8:19写道： > Thanks Till for your great work, also thanks to the whole community! > > Thanks, vino. > > Timo Walther 于2018年11月30日周五 下午7:42写道： > > > Thanks for b

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Piotrek and Jark, Thanks for the feedback and explanation. Those are good arguments. But I think those arguments are mostly about materialized view. Let me try to explain the reason I believe cache() and materialize() are different. I think cache() and materialize() have quite different implic

Re: Custom scheduler in Flink

thanks. I saw the google docs just now. I am gonna print and study it at the weekend. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com * On Fri, Nov 30, 2018 at 4:42 PM Felipe Gutierrez < felipe.o.gutier.

Re: Custom scheduler in Flink

thanks. I will check it out *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com * On Fri, Nov 30, 2018 at 3:51 PM Till Rohrmann wrote: > Hi Felipe, > > https://issues.apache.org/jira/browse/FLINK-10429 mig

Re: Custom scheduler in Flink

Hi Felipe, https://issues.apache.org/jira/browse/FLINK-10429 might also be interesting. The community is currently working on making the Scheduler pluggable to make it easier to extend this component. Cheers, Till On Wed, Nov 28, 2018 at 2:56 PM Felipe Gutierrez < felipe.o.gutier...@gmail.com> w

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Thanks Piotrek, You provided a very good example, it explains all the confusions I have. It is clear that there is something we have not considered in the initial proposal. We intend to force the user to reuse the cached/materialized table, if its cache() method is executed. We did not expect that

Re: Apply for flink contributor permission

Hi Qi, I gave you contributor permissions. Cheers, Till On Wed, Nov 28, 2018 at 3:38 AM qi luo wrote: > Hi Flink team, > > Could you pls give me the contributor permission, as I’m working on JIRA > issue [1]. My Jira ID is *Qi* (luoqi...@gmail.com luoqi...@gmail.com>) > > [1] https://issues.a

[jira] [Created] (FLINK-11043) Website Flink logo broken on Firefox

Patrick Lucas created FLINK-11043: - Summary: Website Flink logo broken on Firefox Key: FLINK-11043 URL: https://issues.apache.org/jira/browse/FLINK-11043 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-11042) testFlinkKafkaProducerFailTransactionCoordinatorBeforeNotify is an invalid test

Piotr Nowojski created FLINK-11042: -- Summary: testFlinkKafkaProducerFailTransactionCoordinatorBeforeNotify is an invalid test Key: FLINK-11042 URL: https://issues.apache.org/jira/browse/FLINK-11042 P

[jira] [Created] (FLINK-11041) ReinterpretDataStreamAsKeyedStreamITCase.testReinterpretAsKeyedStream failed on Travis

Till Rohrmann created FLINK-11041: - Summary: ReinterpretDataStreamAsKeyedStreamITCase.testReinterpretAsKeyedStream failed on Travis Key: FLINK-11041 URL: https://issues.apache.org/jira/browse/FLINK-11041

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Shaoxuan, Re 2: > Table t3 = methodThatAppliesOperators(t1) // t1 is modified to-> t1’ What do you mean that “ t1 is modified to-> t1’ ” ? That `methodThatAppliesOperators()` method has changed it’s plan? I was thinking more about something like this: Table source = … // some source that

[jira] [Created] (FLINK-11040) Incorrect generic type of DataStream in broadcast_state.md

Yangze Guo created FLINK-11040: -- Summary: Incorrect generic type of DataStream in broadcast_state.md Key: FLINK-11040 URL: https://issues.apache.org/jira/browse/FLINK-11040 Project: Flink Issue

Re: [ANNOUNCE] Apache Flink 1.7.0 released

Thanks Till for your great work, also thanks to the whole community! Thanks, vino. Timo Walther 于2018年11月30日周五 下午7:42写道： > Thanks for being the release manager Till and thanks for the great work > Flink community! > > Regards, > Timo > > > Am 30.11.18 um 10:39 schrieb Till Rohrmann: > > The Apa

Re: Looking for relevant sources related to connecting Apache Flink and Edgent.

Cool, thanks! I am able to verify the Execution Query Plan on this example: https://github.com/felipegutierrez/flink-first/blob/master/src/main/java/flink/example/streaming/SocketWindowWordCountFilterJava.java I am also going to build a little POC like you said. Thanks, Felipe *--* *-- Felipe Gut

Re: [ANNOUNCE] Apache Flink 1.7.0 released

Thanks for being the release manager Till and thanks for the great work Flink community! Regards, Timo Am 30.11.18 um 10:39 schrieb Till Rohrmann: The Apache Flink community is very happy to announce the release of Apache Flink 1.7.0, which is the next major release. Apache Flink® is an open

[jira] [Created] (FLINK-11039) LogicalExchange and HashPartitioner realization

ambition created FLINK-11039: Summary: LogicalExchange and HashPartitioner realization Key: FLINK-11039 URL: https://issues.apache.org/jira/browse/FLINK-11039 Project: Flink Issue Type: Improveme

Re: Looking for relevant sources related to connecting Apache Flink and Edgent.

Hi Felipe, You can define TableSources (for SQL, Table API) that support filter push-down. The optimizer will figure out this opportunity and hand filters to a custom TableSource. I should add that AFAIK this feature is not used very often (expect some rough edges) and that the API is likely to c

[ANNOUNCE] Apache Flink 1.7.0 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.7.0, which is the next major release. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. The release is availab

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi, It is an very interesting and useful design! Here I want to share some of my thoughts: 1. Agree with that cache() method should return some Table to avoid some unexpected problems because of the mutable object. All the existing methods of Table are returning a new Table instance. 2. I th

Re: Looking for relevant sources related to connecting Apache Flink and Edgent.

I guess this message from 2016 is very related of what I am looking for ( http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Execution-Plan-td4290.html). I am posting here for future references. I am going to implement a toy example to visualize this. Do you guys see this de

[jira] [Created] (FLINK-11038) Rewrite Kafka at-least-once it cases

Piotr Nowojski created FLINK-11038: -- Summary: Rewrite Kafka at-least-once it cases Key: FLINK-11038 URL: https://issues.apache.org/jira/browse/FLINK-11038 Project: Flink Issue Type: Bug

Re: [DISCUSS] Support Interactive Programming in Flink Table API

Hi Fabian and Piotr, Thanks for the feedback. I think I now understand you a little better. 1. “Materialize" and “cache" are two different scenarios IMO. "Materialize" is a complex feature that allows the user to really create a materializedView/table, and the materialized table will be timely up

[jira] [Created] (FLINK-11037) Introduce another greedy mechanism for distributing floating buffers

zhijiang created FLINK-11037: Summary: Introduce another greedy mechanism for distributing floating buffers Key: FLINK-11037 URL: https://issues.apache.org/jira/browse/FLINK-11037 Project: Flink

[jira] [Created] (FLINK-11036) Streaming classloader end-to-end test does not work on source releases

Timo Walther created FLINK-11036: Summary: Streaming classloader end-to-end test does not work on source releases Key: FLINK-11036 URL: https://issues.apache.org/jira/browse/FLINK-11036 Project: Flink