Re: Flink Dataset to ParquetOutputFormat

Hi Anuj, After searching in Github, I found a demo repository about how to use parquet in Flink.[1] You can have a look. I can not make sure whether it is helpful or not. [1]: https://github.com/FelixNeutatz/parquet-flinktacular Best, Vino aj 于2019年12月21日周六 下午7:03写道： > Hello All, > > I am ge

Re: using thin jar to replace fat jar on yarn cluster mode

Hi, I think you can try specifying dependent jars with the -C option[1] when you submit the job, and see if that meets your needs. [1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/cli.html#usage On Mon, Dec 23, 2019 at 10:09 AM zjfpla...@hotmail.com < zjfpla...@hotmail.com> wrote:

Re: Flink On K8s, build docker image very slowly, is there some way to make it faster?

Hi Lake, Can you clearly count or identify which steps are taking a long time? Best, Vino LakeShen 于2019年12月23日周一 下午2:46写道： > Hi community , when I run the flink task on k8s , the first thing is that > to build the flink task jar to > Docker Image . I find that It would spend much time to buil

Re: Flink On K8s, build docker image very slowly, is there some way to make it faster?

Hi Lake, Usually building a docker image should not take much time (typically less than 2 minutes). It is probably the network issue that causes the long time for image building. Of course we will need more information (e.g., logs) to confirm that, but according to our experience pulling the base

Flink On K8s, build docker image very slowly, is there some way to make it faster?

Hi community , when I run the flink task on k8s , the first thing is that to build the flink task jar to Docker Image . I find that It would spend much time to build docker image. Is there some way to makr it faster. Thank your replay.

Re: Flink task node shut it self off.

Hi John, Thanks for the positive comments of Flink usage. No matter at least-once or exactly-once you used for checkpoint, it would never lose one message during failure recovery. Unfortunatelly I can not visit the logs you posted. Generally speaking the longer internal checkpoint would mean r

Re: Taskmanagers in Docker Fail to Resolve Own Hostnames and Won't Accept Tasks

Hi Martin, Could you `docker exec` into the problematic taskmanager and check whether the hostname could be resolved to a correct ip? You could use `nslookup {tm_hostname}` to verify. Best, Yang Martin, Nick J [US] (IS) 于2019年12月21日周六 上午6:07写道： > I’m running Flink 1.7.2 in a Docker swarm. Int

using thin jar to replace fat jar on yarn cluster mode

Hi, Does flink on yarn support using thin jar to replace fat jar? I don't want the jar of each flink task to have hundreds of MB. I want to put all the dependent packages in a single directory,and then the size of each flink task jar will be tens of KB. __

Re: [DISCUSS] What parts of the Python API should we focus on next ?

Hi Bowen, Your suggestions are very helpful for expanding the PyFlink ecology. I also mentioned above to integrate notebooks，Jupyter and Zeppelin are both very excellent notebooks. The process of integrating Jupyter and Zeppelin also requires the support of Jupyter and Zeppelin community personne

[ANNOUNCE] Weekly Community Update 2019/51

Dear community, Happy to share this week's brief community digest with updates on Flink 1.10 and Flink 1.9.2, a proposal to integrate Flink Docker image publication into Flink release process, a discussion on new features of PyFlink and a couple of blog posts. Enjoy. Flink Development ===

Query on state of ValueState getting lost

Hi, I am using flink 1.8.2, event time stream for sliding window transformation with custom trigger. Window size is 15 seconds, slide size is also 15 seconds with 30 seconds allowed lateness. In the trigger implementation couple of class level ValueStateDescriptors are used, the corresponding Valu