Re: Parallelizing a tumbling group window

2017-12-29 Thread Colin Williams
Hi Timo and flink-user, It's been a few weeks and we've made some changes to the application mentioned on this email. we've also updated for flink 1.4 . We are using the SQL / Table API with a tumbling window and user defined agg to generate a SQL query string like: SELECT measurement, `tag_App

Parquet Format Read and Write

2017-12-29 Thread Imran Tariq
Hi, I am new to Flink and exploring it. Just curious to know that, do Flink API support Parquet format like Spark API can easily read and write in Parquet. -- *Thanks & RegardsMuhammad Imran Tariq*

queryable state and maintaining all time counts

2017-12-29 Thread jelmer
Hi, I've been going through various talks on flink's support for queryable state. Like this talk by Jamie Grier at 2016's Flink forward : https://www.youtube.com/watch?v=uuv-lnOrD0o I see how you can easily use this to produce time series data. Eg calculate the number of events per hour. But I

Re: MergingWindow

2017-12-29 Thread jincheng sun
Hi aitozi, `MergingWindowSet` is a Utility, used for keeping track of merging Windows when using a MergingWindowAssigner in a WindowOperator. In flink `MergingWindowAssigner` only used for SessionWindow. The implementations of `MergingWindowAssigner` are `EventTimeSessionWindows` and `Process

Re: keyby() issue

2017-12-29 Thread Jinhua Luo
I misuse the key selector. I checked the doc and found it must return deterministic key, so using random is wrong, but I still could not understand why it would cause oom. 2017-12-28 21:57 GMT+08:00 Jinhua Luo : > It's very strange, when I change the key selector to use random key, > the jvm rep

Re: How to stop FlinkKafkaConsumer and make job finished?

2017-12-29 Thread Ufuk Celebi
Yes, that sounds like what Jaxon is looking for. :-) Thanks for the pointer Eron. – Ufuk On Thu, Dec 28, 2017 at 8:13 PM, Eron Wright wrote: > I believe you can extend the `KeyedDeserializationSchema` that you pass to > the consumer to check for end-of-stream markers. > > https://ci.apache.org/p