回复：Dataset and eager scheduling

Hi,     From my understand,  if you do not care resource waste and confirm there are enough resources in cluster, you can set EAGER schedule mode for batch job. From optimizer aspect, if not set the PIPELINED_FORCED hint for  ExecutionMode, for some special topology cases, the optimizer would

[jira] [Created] (FLINK-5955) Merging a list of buffered records will have problem when ObjectReuse is turned on

Shaoxuan Wang created FLINK-5955: Summary: Merging a list of buffered records will have problem when ObjectReuse is turned on Key: FLINK-5955 URL: https://issues.apache.org/jira/browse/FLINK-5955 Proj

Re: [DISCUSS] Gelly planning for release 1.3 and roadmap

Hi Visia and Greg, I totally agree with you. The basic design idea behind Flink and Gelly's API meets my personal taste well. Marking stable API must be not easy as it looks like and I don't think I am eligible to talk about it now : ) IMO, updating multiple datasets is essential for making Gell

[jira] [Created] (FLINK-5954) Always assign names to the window in the Stream SQL APi

Haohui Mai created FLINK-5954: - Summary: Always assign names to the window in the Stream SQL APi Key: FLINK-5954 URL: https://issues.apache.org/jira/browse/FLINK-5954 Project: Flink Issue Type: I

Re: About the ACL of Zookeeper

I assume ZK is configured to use Kerberos. I would check few configurations on the ZK side to make sure that Kerberos configurations are working fine.  On the ZK server side configurations, 1) authProvider is configured to use SASL Authentication provider2) ZK server is using approptiate JAAS en

[jira] [Created] (FLINK-5953) Sample Kinesis Scala code doesn't work

Matthew Billson created FLINK-5953: -- Summary: Sample Kinesis Scala code doesn't work Key: FLINK-5953 URL: https://issues.apache.org/jira/browse/FLINK-5953 Project: Flink Issue Type: Bug

Dataset and select/split functionality

Hi all, We will try to implement select/split functionality for batch api. We looked at streaming side and understand how it works but since streaming side does not include an optimizer it was easier. Since adding such a runtime operator will affect optimizer layer as well, is there a part that yo

Re: About the ACL of Zookeeper

Hi! Is the ACL Kerberos-based? If yes, you need to make sure the Kerberos module for ZooKeeper is loaded: https://ci.apache.org/projects/flink/flink-docs-release-1.2/ops/security-kerberos.html https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/jobmanager_high_availability.html#conf

Re: Dataset and eager scheduling

Hi Till, Thank you. On 2 March 2017 at 18:13, Till Rohrmann wrote: > Hi CPC, > > I think that the optimizer does not take the scheduling mode into account > when optimizing a Flink job. > > Cheers, > Till > > On Thu, Mar 2, 2017 at 11:52 AM, CPC wrote: > > > Hi all, > > > > Currently our tea

Re: Dataset and eager scheduling

Hi CPC, I think that the optimizer does not take the scheduling mode into account when optimizing a Flink job. Cheers, Till On Thu, Mar 2, 2017 at 11:52 AM, CPC wrote: > Hi all, > > Currently our team trying implement a runtime operator also playing with > scheduler. We are trying to understan

Re: About the ACL of Zookeeper

Hi Zhangrucong, I don't exactly know what's needed to use the ACL of ZooKeeper. I'm pulling Vijay in who implemented this feature. He probably knows more about it. Cheers, Till On Thu, Mar 2, 2017 at 3:09 AM, Zhangrucong wrote: > Hi: > > I want to use the ACL of Zookeeper. So I configure

[jira] [Created] (FLINK-5952) JobCancellationWithSavepointHandlersTest uses deprecated JsonNode#getValuesAsText

Chesnay Schepler created FLINK-5952: --- Summary: JobCancellationWithSavepointHandlersTest uses deprecated JsonNode#getValuesAsText Key: FLINK-5952 URL: https://issues.apache.org/jira/browse/FLINK-5952

Re: [DISCUSS] Side Outputs and Split/Select

+1 2017-03-02 12:11 GMT+01:00 Aljoscha Krettek : > Ok, so it seems we have to go with the OutputTag variant for windows as > well, for now. > > For Flink 2.0 we can change that. Would everyone be OK with that? > > On Thu, Mar 2, 2017 at 12:05 PM, Robert Metzger > wrote: > > > Flink enforces bina

Re: [DISCUSS] Side Outputs and Split/Select

Ok, so it seems we have to go with the OutputTag variant for windows as well, for now. For Flink 2.0 we can change that. Would everyone be OK with that? On Thu, Mar 2, 2017 at 12:05 PM, Robert Metzger wrote: > Flink enforces binary compatibility for all classes tagged with the @Public > annotat

Re: [DISCUSS] Side Outputs and Split/Select

Flink enforces binary compatibility for all classes tagged with the @Public annotation. Binary compatibility allows users to execute a job against a newer Flink version without recompiling their job jar. Your change alters the return type of some methods (apply()). I think there's no way to do that

Dataset and eager scheduling

Hi all, Currently our team trying implement a runtime operator also playing with scheduler. We are trying to understand batch optimizer but it will take some time. What we want to know is whether changing batch scheduling mode from LAZY_FROM_SOURCES to EAGER could affect optimizer? I mean whether

[jira] [Created] (FLINK-5951) Register legacy windowOp timers with timer service in the restoreState().

Kostas Kloudas created FLINK-5951: - Summary: Register legacy windowOp timers with timer service in the restoreState(). Key: FLINK-5951 URL: https://issues.apache.org/jira/browse/FLINK-5951 Project: Fl

Re: [DISCUSS] Flink ML roadmap

Hi All, First of all I'd like to introduce myself: my name is Roberto Bentivoglio and I'm currently working for Radicalbit as Andrea Spina (he already wrote on this thread). I didn't have the chance to directly contribute on Flink up to now, but some colleagues of mine are doing that since at leas

[jira] [Created] (FLINK-5950) create an example for async-io with an http client

David Anderson created FLINK-5950: - Summary: create an example for async-io with an http client Key: FLINK-5950 URL: https://issues.apache.org/jira/browse/FLINK-5950 Project: Flink Issue Type