Re: [ANNOUNCE] Apache Flink 1.11.0 released

2020-07-07 Thread Wesley
Nice news. Congrats! Leonard Xu wrote: Congratulations! Thanks Zhijiang and Piotr for the great work, and thanks everyone involved! Best, Leonard Xu

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

2019-09-11 Thread Wesley Peng
Hi on 2019/9/11 17:22, Till Rohrmann wrote: I'm very happy to announce that Zili Chen (some of you might also know him as Tison Kun) accepted the offer of the Flink PMC to become a committer of the Flink project. Congratulations Zili Chen. regards.

Re: [DISCUSS] Drop older versions of Kafka Connectors (0.9, 0.10) for Flink 1.10

2019-09-11 Thread Wesley Peng
on 2019/9/11 16:17, Stephan Ewen wrote: We still maintain connectors for Kafka 0.8 and 0.9 in Flink. I would suggest to drop those with Flink 1.10 and start supporting only Kafka 0.10 onwards. Are there any concerns about this, or still a significant number of users of these versions? Bu

Re: [flink-1.9] how to read local json file through Flink SQL

2019-09-08 Thread Wesley Peng
On 2019/9/8 5:40 下午, Anyang Hu wrote: In flink1.9, is there a way to read local json file in Flink SQL like the reading of csv file? hi, might this thread help you? http://mail-archives.apache.org/mod_mbox/flink-dev/201604.mbox/%3cCAK+0a_o5=c1_p3sylrhtznqbhplexpb7jg_oq-sptre2neo...@mail.gmail.

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

2019-09-06 Thread Wesley Peng
On 2019/9/6 8:55 下午, Fabian Hueske wrote: I'm very happy to announce that Kostas Kloudas is joining the Flink PMC. Kostas is contributing to Flink for many years and puts lots of effort in helping our users and growing the Flink community. Please join me in congratulating Kostas! congratulat

Re: flink sql row_number() over () OOM

2019-09-04 Thread Wesley Peng
Hi on 2019/9/4 19:30, liu ze wrote: I use the row_number() over() function to do topN, the total amount of data is 60,000, and the state is 12G . Finally, oom, is there any way to optimize it? ref: https://stackoverflow.com/questions/50812837/flink-taskmanager-out-of-memory-and-memory-config

Re: Compress DataSink Output

2016-08-19 Thread Wesley Kerr
hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.html > > Let me know if you need more information. > > Regards, > Robert > > [1]: > https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/hadoop_compatibility.html &g

Compress DataSink Output

2016-08-17 Thread Wesley Kerr
Hello - Forgive me if this has been asked before, but I'm trying to determine the best way to add compression to DataSink Outputs (starting with TextOutputFormat). Realistically I would like each partition file (based on parallelism) to be compressed independently with gzip, but am open to other