Question about configuring Rich Functions

This document: https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/best_practices.html#parsing-command-line-arguments-and-passing-them-around-in-your-flink-application Apache Flink 1.3 Documentation: Best Practices

Re: Question about Global Windows.

😊 https://issues.apache.org/jira/browse/FLINK-7473 From: Steve Jerman Sent: Thursday, August 17, 2017 11:34:09 AM To: Nico Kruber; user@flink.apache.org Subject: Re: Question about Global Windows. Thank you Nico. I *think* I should have one stream per key

Re: Question about Global Windows.

r thoughts Steve From: Nico Kruber Sent: Wednesday, August 16, 2017 3:22:41 AM To: user@flink.apache.org Cc: Steve Jerman Subject: Re: Question about Global Windows. Hi Steve, are you sure a GlobalWindows assigner fits your needs? This may be the case if all y

Question about Global Windows.

Hi Folks, I have a question regarding Global Windows. I have a stream with a large number of records. The records have a key which has a very high cardinality. They also have a state ( start, status, finish). I need to do some processing where I look at the records separated into windows using

Advice on debugging state back end...

Hi Folks, I have a stream application which is running out of heap space - looks like there might be something up with state storage I'm having trouble determining if it just needs a lot of memory or there is a memory leak. Are there any tips/best practice for this? I've looked at heap du

Re: Quick Question...

g your transformations, something along the lines of dataset.map(..).withConfiguration. In the DataStream API you cannot set the Configuration at all. Note that in both APIs you can also just pass the Configuration into the constructor and store it in a field. Regards, Chesnay On 22.06.2017 19:56,

Quick Question...

Hi, I have a quick question… How do I set the Configuration passed into RichFunction.open? I *thought* that setting GlobalJobParameters would do it ... env.getConfig().setGlobalJobParameters(jobParameters); But it seems not… Steve

Re: Checkpoints?

val is shorter than this time, then the system won't be able to delete old checkpoints quick enough. Cheers, Till On Fri, Jun 2, 2017 at 5:57 PM, Steve Jerman mailto:st...@kloudspot.com>> wrote: Hi, Configuration: Flink 1.2.0 I'm using the Rocks DB backend for checkpointing. The

Checkpoints?

Hi, Configuration: Flink 1.2.0 I'm using the Rocks DB backend for checkpointing. The problem I have is that no checkpoints are being deleted, and my disk is filling up. Is there configuration for this? Thanks Steve

Using FlinkML from Java?

Hi Folks, I’m trying to use FlinkML 1.2 from Java … getting this: SVM svm = new SVM() .setBlocks(env.getParallelism()) .setIterations(100) .setRegularization(0.001) .setStepsize(0.1) .setSeed(42); svm.fit(labelledTraining); The type org.apache.flink.api.scala.DataSet cannot be resolv

Re: In-Memory data grid

I had a DB look up to do and used com.google.common.cache.Cache with a 10 sec timeout: private static Cache locationCache = CacheBuilder.newBuilder().maximumSize(1000).expireAfterAccess(10, TimeUnit.SECONDS).build(); Seemed to help a lot with throughput…. Steve On Apr 3, 2017, at 10:55 AM,

Question Regarding a sink..

Hi, I have a sink writing data to InfluxDB. I’ve noticed that the sink gets multiple copies of upstream records.. Why does this happen, and how can I avoid it… ? Below is a trace …showing 2 records (I have a parallelism of two) for each record in the ‘.printToError’ for the same stream. Any h

Re: How to use 'dynamic' state

worked a lot on the stuff mentioned here. Cheers, Gordon [1] FLIP-17: https://cwiki.apache.org/confluence/display/FLINK/FLIP-17+Side+Inputs+for+DataStream+API On March 7, 2017 at 5:05:04 AM, Steve Jerman (st...@kloudspot.com<mailto:st...@kloudspot.com>) wrote: I’ve been reading t

How to use 'dynamic' state

I’ve been reading the code/user goup/SO and haven’t really found a great answer to this… so I thought I’d ask. I have a UI that allows the user to edit rules which include specific criteria for example trigger event if X many people present for over a minute. I would like to have a flink job th