Re: MODERATE for d...@flink.apache.org

Dear Flink Dev Team I have encountered a problem and can't find any solution in Google. And I have created a thread in stackoverflow.com but no response. https://stackoverflow.com/questions/47123371/flink-windows-ha I would really appreciate it if you could give some suggestions to me. Thanks,

Re: DataStream to Table Api idioms

Not a problem, thanks for the quick feedback. https://issues.apache.org/jira/browse/FLINK-7999 Seth Wiesman From: Fabian Hueske Date: Monday, November 6, 2017 at 9:14 AM To: Seth Wiesman Cc: user Subject: Re: DataStream to Table Api idioms Hi Seth, I think the Table API is not there yet to

Re: Savepoints and migrating value state data types

Actually, Flink 1.4 will come with improved Avro support. See especially: - https://issues.apache.org/jira/browse/FLINK-7420: Move All Avro Code to flink-avro - https://issues.apache.org/jira/browse/FLINK-7997:

Re: DataStream to Table Api idioms

Hi Seth, I think the Table API is not there yet to address you use case. 1. Allowed lateness cannot be configured but it is on the list of features that we plan to add in the future. 2. Custom triggers are not supported. We are planning to add an option to support your use case (early firing and

Re: Savepoints and migrating value state data types

Hi Gordon I've been looking into creating a custom AvroSerializer without Kryo which would support Avro schemas and I'm starting to wonder if this is actually the most straightforward way to do it. If I extend a class from TypeSerializer I would also need to implement a TypeInformation class to

DataStream to Table Api idioms

Hi, I am experimenting with rewriting some of my datastream projects with the table api and I had some questions on how to express certain idioms. I am using 1.4-SNAPSHOT. 1) Can I express allowed lateness? 2) Can I use a custom trigger? More specifically, I have a 24hr window bu

Re: Docker-Flink Project: TaskManagers can't talk to JobManager if they are on different nodes

I'm not entirely sure how docker swarm works but from the Flink perspective there mustn't be two TaskManagers running on the same host (meaning an entity where you share the same address) if you set the TaskManager data port to a fixed value (otherwise only one of them can be started due to port co

Re: Docker-Flink Project: TaskManagers can't talk to JobManager if they are on different nodes

Hi Till, Thanks a lot for your answer. Is the taskmanager.data.port unique per TaskManager? The documentation says it is assigned at runtime by the OS. My thinking here is that you would need to know what that is at service creation time, which would go against the whole idea of how services

Re: JobManager web interface redirect strategy when running in HA

Chesnay, your solution is definitely the best approach. I was already wondering why the decision was made to only support the UI through the leading job manager only. Jürgen, I don't think that your solution will work in our setup. We're currently running 3 services, one for each job manager. We n

Re: Docker-Flink Project: TaskManagers can't talk to JobManager if they are on different nodes

Hi Thalita, in order to make Flink work, I think you have to expose the JobManager RPC port, the Blob server port and make sure that the TaskManager can talk to each other by exposing the `taskmanager.data.port`. The query server port is only necessary if you want to use queryable state. I've pul

FlinkCEP behaviour with time constraints not as expected

Hi everyone, I wanted to ask if FlinkCEP in the following scenario is working as it should, or I have misunderstood its functioning. I've got a keyedstream associated with the following pattern: Pattern[Event].begin("start").where(_.value >=100).oneOrMore .notNext("end").where(_.value >=100).wit

Re: Negative values using latency marker

Ok, digging into it a bit further: The LatencyMarker is scheduled at a certain period with some initialDelay. Its initial time is `System.currentTimeMillis() + initialDelay` (when it should first be run). Depending on your system's load, this run may actually be delayed (but then the marker's t

Re: Facing issues with Logback

Hi Teena, thanks for reaching out to the mailing list for this issue. This sound indeed like a bug in Flink and should be investigated. We are currently working on a new release 1.4 and the testing phase will start soon. So it would make sense to include this problem in the testing and hopefully i

Re: Docker-Flink Project: TaskManagers can't talk to JobManager if they are on different nodes

Till, is there somewhere a list of ports that need to exposed that’s more up to date compared to docker-flunk README? Piotrek > On 3 Nov 2017, at 10:23, Vergilio, Thalita > wrote: > > Just an update: by changing the JOB_MANAGER_RPC_ADDRESS to the public IP of > the JobManager and exposing po