Re: Too many open files on Bucketing sink

2018-03-14 Thread Felix Cheung
I have seen this before as well. My workaround was to limit the number of parallelism but it is the unfortunate effect of limiting the number of processing tasks also (and so slowing things down) Another alternative is to have bigger buckets (and smaller number of buckets) Not sure if there is

Re: [EXTERNAL] Re: Bucketing/Rolling Sink: New timestamp appeded to the part file name everytime a new part file is rolled

2017-09-01 Thread Felix Cheung
Yap I was able to get this to work with a custom bucketer. A custom bucketer can use the clock given ("processing time") or it can use a timestamp from the data ("event time") for the bucketing path. From: Raja.Aravapalli Sent: Friday, September 1, 2017 10:21:00

Is there a way to change the port for the Flink monitoring page?

2015-09-16 Thread Felix Cheung
I'm  using Flink from Zeppelin in local mode and Zeppelin is using 8080