Re: Two issues when deploying Flink on DC/OS

2018-02-08 Thread Stavros Kontopoulos
We see the same issue here (2): 2018-02-08 10:55:11,447 ERROR org.apache.flink.runtime.webmonitor.files.StaticFileServerHandler - Caught exception java.io.IOException: Connection reset by peer Stavros On Sat, Jan 13, 2018 at 9:59 PM, Eron Wright wrote: > Hello Dongwon, > > Flink doesn't suppor

Re: In-memory cache

2017-10-02 Thread Stavros Kontopoulos
How about connecting two streams of data, one from the reference data and one from the main data (I assume using key streams as you mention QueryableState) and keep state locally within the operator. The idea is to have a local sub-copy of the reference data within the operator that is updated from

Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-10 Thread Stavros Kontopoulos
Congrats! On Fri, Feb 10, 2017 at 9:11 PM, Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Congrats! > > On 2/10/17 2:00 AM, Ufuk Celebi wrote: > > Hey everyone, > > > > I'm very happy to announce that the Flink PMC has accepted Stefan > > Richter to become a commi

subtasks and objects

2016-05-24 Thread Stavros Kontopoulos
Hey, Is it true that since taskmanager (a jvm) may have multiple subtasks implementing the same operator and thus same logic and loading the same classes, no separating classloading is done right? So if i use a scala object or static code as in java within that logic then that is shared among the

Re: flink async snapshots

2016-05-19 Thread Stavros Kontopoulos
h-talk-google-on-flink-fault-tolerance-and-ha > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> > > On 19 May 2016, at 19

Re: flink snapshotting fault-tolerance

2016-05-19 Thread Stavros Kontopoulos
threshold > exists in the Flink configuration. > > On 19 May 2016, at 20:42, Stavros Kontopoulos > wrote: > > The problem here is different though if something is keep failing > (permanently) in practice someone needs to be notified. If the user loses > snapshotting he must kn

Re: flink snapshotting fault-tolerance

2016-05-19 Thread Stavros Kontopoulos
.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> > > O

Re: flink snapshotting fault-tolerance

2016-05-19 Thread Stavros Kontopoulos
to the driver? On Thu, May 19, 2016 at 9:30 PM, Paris Carbone wrote: > In that case, typically a timeout invalidates the whole snapshot (all > states for the same epoch) until eventually we have a full complete > snapshot. > > > On 19 May 2016, at 20:26, Stavros Ko

Re: flink snapshotting fault-tolerance

2016-05-19 Thread Stavros Kontopoulos
slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> &

Re: flink snapshotting fault-tolerance

2016-05-19 Thread Stavros Kontopoulos
olerance-and-ha > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> > > > <http://www.slideshare.net/ParisCarbone/tech-talk-google-on-flink-fault-tolerance-and-ha> > > On 19 May 2016, at 19:43, Stavros Kontopoulos &

Re: flink snapshotting fault-tolerance

2016-05-19 Thread Stavros Kontopoulos
tem for a certain period of time, the checkpoint is cancelled. The > default timeout is 10 minutes. > > Regards, > Robert > > > On Mon, May 16, 2016 at 1:22 PM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > >> Hi, >> >> I was looking into

flink snapshotting fault-tolerance

2016-05-16 Thread Stavros Kontopoulos
Hi, I was looking into the flink snapshotting algorithm details also mentioned here: http://data-artisans.com/high-throughput-low-latency-and-exactly-once-stream-processing-with-apache-flink/ https://blog.acolyer.org/2015/08/19/asynchronous-distributed-snapshots-for-distributed-dataflows/ http://m

Re: withBroadcastSet for a DataStream missing?

2016-05-08 Thread Stavros Kontopoulos
ation. > > Cheers, > Till > ​ > > On Sun, Apr 17, 2016 at 7:16 PM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > >> Im trying what you suggested. Is this what you are suggesting (this is >> just a skeleton of logic not the actual implementation)? &

Re: withBroadcastSet for a DataStream missing?

2016-04-17 Thread Stavros Kontopoulos
the clustered points and model updates. Here you have to use the split >> and select function to split the output stream into model updates and >> output elements. It’s important to broadcast the model updates, otherwise >> not all operators have the same clustering model. >>

Re: withBroadcastSet for a DataStream missing?

2016-04-01 Thread Stavros Kontopoulos
ut elements. It’s important to broadcast the model updates, otherwise > not all operators have the same clustering model. > > Cheers, > Till > ​ > > On Tue, Mar 29, 2016 at 7:23 PM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > >> H i am new here

withBroadcastSet for a DataStream missing?

2016-03-29 Thread Stavros Kontopoulos
H i am new here... I am trying to implement online k-means as here https://databricks.com/blog/2015/01/28/introducing-streaming-k-means-in-spark-1-2.html with flink. I dont see anywhere a withBroadcastSet call to save intermediate results is this currently supported? Is intermediate results state