State in flink jobs

Hi, Is the value state persisted across job restart/deployment without savepoint if checkpointing is disabled.

Measuring Execution Time

Hi All, I have a program that executes multiple (data dependent) iterative jobs one after another in the same execution plan. It has the following structure ExecutionEnvironment env = .. LoadData Iterative Process 1 Iterative Process 2 ... Iterative Process N env.execute("...") I wan

Restart the job from a checkpoint

Hi team, Can we restart a flink job from previous successful checkpoint? I know we can start a flink from a savepoint but I wonder if I can do it similar by passing the checkpoint path to the flink run command to restore the job from checkpoint.

Re: Kafka KeyedStream source

Hi Niels, If it’s only for simple data filtering that does not depend on the key, a simple “flatMap” or “filter" directly after the source can be chained to the source instances. What that does is that the filter processing will be done within the same thread as the one fetching data from a Kaf

Re: Deduplicate messages from Kafka topic

Hi, You’re correct that the FlinkKafkaProducer may emit duplicates to Kafka topics, as it currently only provides at-least-once guarantees. Note that this isn’t a restriction only in the FlinkKafkaProducer, but a general restriction for Kafka's message delivery. This can definitely be improved t

Re: 答复: [DISCUSS] Apache Flink 1.2.0 RC0 (Non-voting testing release candidate)

I found two problematic issues with Mesos HA mode which breaks it: https://issues.apache.org/jira/browse/FLINK-5495 https://issues.apache.org/jira/browse/FLINK-5496 On Fri, Jan 13, 2017 at 11:29 AM, Fabian Hueske wrote: > I tested the Table API / SQL a bit. > > I implemented a windowed aggregat

Re: Apache Flink 1.1.4 - Java 8 - CommunityDetection.java:158 - java.lang.NullPointerException

Hi Miguel, this is a bug, thanks a lot for reporting! I think the problem is that the implementation assumes that labelsWithHighestScores contains the vertex itself as initial label. Could you please open a JIRA ticket for this and attach your code and data as an example to reproduce? We should a

Re: Terminology: Split, Group and Partition

Hi Robert, thanks for opening the ticket. Regarding injecting grouping or partitioning information, semantic annotations (forward fields) [1] is probably what you are looking for. Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/batch/index.html#semantic-annotat

Re: some questions about submit flink job on flink-yarn

Hi Huang, the reason why you cannot use the IP address to send messages to your YARN JobManager is that we no longer resolve the hostname into an IP address. Instead we start the ActorSystem with the unresolved hostname. You can see this in the following log line: `Actor system bound to hostname 9

Re: Flink on YARN: Cannot connect to JobManager

Hi Malte, can it be that you’re trying to request more resources from your yarn cluster than there are currently available? It depends a little bit on your other settings but -yn 2 says that you request 2 TaskManagers. Additionally, Flink will also allocate another container for the JobManager. Pe