Failed to submit 0.10.1

Hi All I’m new to flink and come to the step to submit to a remote cluster, and it failed with following message: Association with remote system [akka.tcp://flink@127.0.0.1:61231] has failed, address is now gated for [5000] ms. Reason is: [scala.Option; local class incompatible: stream classde

Re: Failed to submit 0.10.1

. > > Best, > Max > > On Mon, Feb 8, 2016 at 10:30 AM, Andrew Ge Wu wrote: >> Hi All >> >> I’m new to flink and come to the step to submit to a remote cluster, and it >> failed with following message: >> >> Association with remote system [akk

Re: Failed to submit 0.10.1

Yes, found a special dependency for 2.11, Thanks! org.apache.flink flink-streaming-java_2.11 ${apache.flink.versin} Andrew > On 08 Feb 2016, at 14:18, Andrew Ge Wu wrote: > > Thanks Max > > My local and remote environment are running: Sc

Very old dependencies and solutions

Hi guys, You probably have noticed. I found a lot of old dependencies (http component 3.1/apache configuration 1.6 etc..) in Flink and leads up errors to stuff like this: java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.conn.ssl.SSLConnectionSocketFactory Is there an

Re: Very old dependencies and solutions

"shade" your dependencies in your fat > jar. Shading relocates the classes (and rewrites your code) so the > dependencies do not conflict, but co-exist in different namespaces. > > Greetings, > Stephan > > > On Thu, Feb 18, 2016 at 1:56 PM, Andrew Ge Wu <ma

state.backend.fs.checkpointdir setting

Hi All, I have been experiencing an error stopping my HA standalone setup. The cluster startup just fine, but when i deploy an application to it, I got this exception: java.lang.Exception: Call to registerInputOutput() of invokable failed at org.apache.flink.runtime.taskmanager.Task.ru

Re: state.backend.fs.checkpointdir setting

u have multiple machines, or only one? > Did you copy the configuration to all machines? > > > > On Fri, Feb 19, 2016 at 6:08 PM, Andrew Ge Wu <mailto:andrew.ge...@eniro.com>> wrote: > Hi All, > > I have been experiencing an error stopping my HA standalone setup

Java 8 and keyBy in 1.0.0

Hi, This is not very obvious and looks like a bug. I have a lambda expression to get key from objects in stream: This works: stream.keyBy(value -> value.getId()) This does not: stream.keyBy(myClass::getId) Exception: Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1

Re: Java 8 and keyBy in 1.0.0

< >> <mailto:ches...@apache.org>ches...@apache.org <mailto:ches...@apache.org>> >> wrote: >> based on https://issues.apache.org/jira/browse/FLINK-3138 >> <https://issues.apache.org/jira/browse/FLINK-3138> this is not supported for >> non-stat

Limit buffer size for a job

Hi guys We have a prioritized queue, where high priority item can jump the queue and we do not want to cache too much record in the buffer. Is there a way to configure my streaming source to use less buffer? so source always fetch and get latest high prio records? Any suggestion? thanks! Andr

Re: Limit buffer size for a job

> <https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/index.html#controlling-latency> > > Does this help? > > – Ufuk > > On Thu, Apr 7, 2016 at 3:04 PM, Andrew Ge Wu <mailto:andrew.ge...@eniro.com>> wrote: > > Hi guys > > >

Parallel execution on AllWindows

Hi, I have such task that I want to count window on a stream and execute them batch by batch. Execute a count window may take some time, so I want it to be executed in parallel. I read this part in the documentation when I found it automatically reduced parallelization to 1 * Note: This operat

Re: Parallel execution on AllWindows

ces. Cheers, Aljoscha On Wed, 3 Aug 2016 at 05:10 Andrew Ge Wu wrote: Hi, I have such task that I want to count window on a stream and execute them batch by batch. Execute a count window may take some time, so I want it to be executed in parallel. I read this part in the documentation w

Re: Parallel execution on AllWindows

nation between the parallel instances that would allow it to fire once > 100 elements are received across the parallel instances. > > Cheers, > > Aljoscha > > On Wed, 3 Aug 2016 at 05:10 Andrew Ge Wu <mailto:andrew.ge...@eniro.com>> wrote: > >> Hi, >> >

Release notes 1.1.0?

Hi, We found out there is a new stable version released: 1.1.0 but we can not find any release note. Do anyone know where to find it? We are experience some change of behavior, I’m not sure if it is related. Thanks Andrew -- Confidentiality Notice: This e-mail transmission may contain con

Re: Release notes 1.1.0?

/news/2016/08/08/release-1.1.0.html>, > http://flink.apache.org/blog/release_1.1.0-changelog.html > <http://flink.apache.org/blog/release_1.1.0-changelog.html> > > What does the chart say? Are the results different? is Flink faster or slower > now? > > > Regards,

Re: Release notes 1.1.0?

t; > On Tue, Aug 9, 2016 at 11:50 AM, Andrew Ge Wu <mailto:andrew.ge...@eniro.com>> wrote: > Hi Robert > > > Thanks for the quick reply, I guess I’m one of the early birds. > Yes, it is much slower, I’m not sure why, I copied slaves, masters, > log4j.properties and fl

Re: Release notes 1.1.0?

016 at 11:50 AM, Andrew Ge Wu <mailto:andrew.ge...@eniro.com>> wrote: > Hi Robert > > > Thanks for the quick reply, I guess I’m one of the early birds. > Yes, it is much slower, I’m not sure why, I copied slaves, masters, > log4j.properties and flink-conf.yaml directly f

Re: Release notes 1.1.0?

Oh sorry missed that part, no, Im not explicitly set that. > On 09 Aug 2016, at 15:29, Aljoscha Krettek wrote: > > Hi, > are you setting a StreamTimeCharacteristic, i.e. > env.setStreamTimeCharacteristic? > > Cheers, > Aljoscha > > On Tue, 9 Aug 2016 at 14:52 A

Re: Release notes 1.1.0?

uld > help us better understand which operators are used exactly. (You can of > course remove any security sensitive stuff.) > > Cheers, > Aljoscha > > On Tue, 9 Aug 2016 at 15:30 Andrew Ge Wu <mailto:andrew.ge...@eniro.com>> wrote: > Oh sorry missed that part

Re: Release notes 1.1.0?

(++key >= 24) { > key = 0; > } > return key; > } > }) > .timeWindow(Time.milliseconds(100)) > .apply(...) > > > Sorry for the inconvenience! > > Greetings, > Stephan > > > > On Wed, Au

Re: Release notes 1.1.0?

gt;= 24) { > key = 0; > } > return new Tuple2<>(key, data); > } > }) > .keyBy(0) > .timeWindow(Time.milliseconds(100)) > .apply(...) > > > Greetings, > Stephan > > > > On Wed, Aug 10, 2016

Evolution algorithm on flink

Hi guys I just published my code to maven central, open source ofc. I try to make this as generic as possible. If you are interested, please try it out, and help me to improve this! https://github.com/CircuitWall/machine-learning Thanks! And

automatically submit a job to a HA cluster

Hi, We have a streaming job wants to submit to a HA cluster via jenkins. Recently we had a downtime on one of the master node and we have it restarted, it seems the backup master became master and submitting to the original master does not do anything. Currently we are using command line to ca

Can not stop cluster gracefully

Hi all, You may hit this problem before, from time to time when i ran the stop-cluster script, I get this > No taskmanager daemon to stop on host app25 > No taskmanager daemon to stop on host app26 > No taskmanager daemon to stop on host app27 > No taskmanager daemon to stop on host app83 > No

Re: Can not stop cluster gracefully

looks for the same name again. > > -Max > > > On Tue, Nov 22, 2016 at 11:27 AM, Andrew Ge Wu wrote: >> Hi all, >> >> >> You may hit this problem before, from time to time when i ran the >> stop-cluster script, I get this >> >>> No taskman

Cluster failure after zookeeper glitch.

Hi, We recently had several zookeeper glitch, when that happens it seems to take flink cluster with it. We are running on 1.03 It started like this: 2017-01-19 11:52:13,047 INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessi

Re: Cluster failure after zookeeper glitch.

your configuration and the ZK problem. > > Best, > Stefan > >> Am 19.01.2017 um 13:16 schrieb Andrew Ge Wu : >> >> Hi, >> >> >> We recently had several zookeeper glitch, when that happens it seems to take >> flink cluster with it. >&