Variable time gap in Session Window

Hi, Is it possible to have a variable value inside the time field for session window? input .keyBy() .window(ProcessingTimeSessionWindows.withGap(Time.minutes(10))) .(); In the above code, instead of the value 10, which will be the same for all records, is it possible to have a valu

Regarding Task Slots allocation

Hi, I am trying to deploy a flink job to remote cluster using remoteExecutionEnvironment, I have specified the number of task slots for Task Manager to 2 , so it should have ideally taken 2 slots only, however all the slots are getting utilized. Is there any other configuration I have to do ? Reg

Question about the need of consumer groups from kafka

Hi, If flink can manage kafka without using its own's mechanism (kafka consumers) it still need the consumer group defined from kafka? And why the examples is needed to use the plugin for shadding creating a fat-jar? Thanks

RE: How to find correct import packages ?

Well, you must always "google it" before asking in the lists. https://stackoverflow.com/questions/31211842/any-way-or-shortcut-to-auto-import-the-classes-in-intellij-idea-like-in-eclips Also, if you wanna add a dependency in pom.xml (which is essentially the sceleton of your flink program, that

RE: How to find correct import packages ?

Hello I use (but it is not familiar) IntelliJ IDEA, but there are many settings. What is the exactly place for enabling the auto-completion imports ? Esa -Original Message- From: m@xi [mailto:makisnt...@gmail.com] Sent: Friday, February 16, 2018 12:42 PM To: user@flink.apache.org Subje

Re: Keyby multiple elements

Yes, you can do that. You would have to define a custom trigger. Alternatively, you can also generate more conservative watermarks. That would have the same effect. Best, Fabian 2018-02-16 12:25 GMT+01:00 Björn Zachrisson : > Hi Fabian, > > It does not since my events are out of order within a c

Re: Keyby multiple elements

Hi Fabian, It does not since my events are out of order within a certain interval and removing allowedLateness reduces the elements processed with 99.5%. Is it possible to trigger the window first when the allowed latness value has been passed? Regards Björn Zachrisson On 16 February 2018 at 12

Re: Keyby multiple elements

Hi Björn, You configured an allowed lateness, so this might be caused by late arriving data. In case a late record is received within the allowed lateness, the window function will fire again for the same window. Does that also happen if you remove the allowedLateness() call? Best, Fabian 2018-

Keyby multiple elements

Hi, I have something weird probably a user error :) I'm running a keyby on multiple elements val keyedStream = nonKeyedStream .keyBy(m => (m.1, m.2, m.3.getOrElse(-1), m.4)) then apply a window function val appliedWindow = keyedStream .timeWindow(minutes(WindowTimeMinutes)) .allowedLaten

Re: How to find correct import packages ?

Hello Esa, First I would recommend to use Intellij as your IDE. There you may enable auto-completion imports that are needed, or you can select manually from all the available options of imports. Furthermore, the Apache Flink Documentation page, [1], is not that thorough in many aspects, but it i

How to find correct import packages ?

Hi I am newbie with Flink, Maven and Scala. What is the best way to find the correct imports packages ? (There are not always in documentation ? at least not very clearly) For example I did not find what "import package" I should use in readCsvFile() ? Also something needs to be added pom.xml

Sessions online for #flinkforward SF. Register with 25% discount code.

Hi Flink Community, On behalf of the data Artisans team, I’d like to announce that the sessions for Flink Forward San Francisco are now online! Check out the great lineup of speakers from companies such as American Express, Comcast, Capital One, eBay, Google, Lyft, Netflix, Uber, Yelp, and others