Re: Cherry picking FLINK-9753 and FLINK-9753 on release-1.6

I would be in favor of including FLINK-9903 and FLINK-9753 in Flink 1.6 because they add more value for the user at a comparable small risk of breaking the system since it is a connector. So +1 for cherry picking these two commits. Cheers, Till On Sat, Jul 21, 2018 at 6:16 PM Tony Wei wrote: >

Re: Write bulks files from streaming app

For what it's worth, in Flink directly we found that this pattern is generally not a well working one: windowing data in large windows in order to perform large bulk writes. Instead, the sinks (to file systems) continuously write (possibly across different destination files) files, ensure persiste

[jira] [Created] (FLINK-9907) add CRC32 checksum in table Api and sql

xueyu created FLINK-9907: Summary: add CRC32 checksum in table Api and sql Key: FLINK-9907 URL: https://issues.apache.org/jira/browse/FLINK-9907 Project: Flink Issue Type: New Feature Compo

[jira] [Created] (FLINK-9908) Inconsistent state of SlotPool after ExecutionGraph cancellation

Till Rohrmann created FLINK-9908: Summary: Inconsistent state of SlotPool after ExecutionGraph cancellation Key: FLINK-9908 URL: https://issues.apache.org/jira/browse/FLINK-9908 Project: Flink

[jira] [Created] (FLINK-9909) Remove cancellation of input futures from ConjunctFutures

Till Rohrmann created FLINK-9909: Summary: Remove cancellation of input futures from ConjunctFutures Key: FLINK-9909 URL: https://issues.apache.org/jira/browse/FLINK-9909 Project: Flink Issue

[jira] [Created] (FLINK-9910) Non-queued scheduling failure sometimes does not return the slot

Till Rohrmann created FLINK-9910: Summary: Non-queued scheduling failure sometimes does not return the slot Key: FLINK-9910 URL: https://issues.apache.org/jira/browse/FLINK-9910 Project: Flink

[jira] [Created] (FLINK-9911) SlotPool#failAllocation is called outside of main thread

Till Rohrmann created FLINK-9911: Summary: SlotPool#failAllocation is called outside of main thread Key: FLINK-9911 URL: https://issues.apache.org/jira/browse/FLINK-9911 Project: Flink Issue

[jira] [Created] (FLINK-9912) Release TaskExecutors from SlotPool if all slots have been removd

Till Rohrmann created FLINK-9912: Summary: Release TaskExecutors from SlotPool if all slots have been removd Key: FLINK-9912 URL: https://issues.apache.org/jira/browse/FLINK-9912 Project: Flink

Flink Docker 1.5.x local mode

I tried to run the 1.5.0 and 1.5.1 docker containers with local mode, following the instructions at https://docs.docker.com/samples/library/flink/#run-a-flink-local-cluster docker run --name flink_local -p 8081:8081 -t flink local Starts the cluster, but it does not start a task manager. That's

Re: Write bulks files from streaming app

Here is a pseudocode (sorry) of what I am doing right now: PCollection> writtenFiles = dataStream .withFixedWindow( duration = 1H, trigger = AfterWatermark.pastEndOfWindow() .withLateFirings(AfterFirst.of( AfterPane.elementCountAtLeast(lateCount), Af

Re: Flink Docker 1.5.x local mode

Hi Thomas, I watched the code of docker flink v1.5[1], it just starts job manager component when you run with local mode. More information you can refer: Flink official documentation of docker deployment : https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/deployment/docker.html (out

[jira] [Created] (FLINK-9913) Improve output serialization only once in RecordWriter

zhijiang created FLINK-9913: --- Summary: Improve output serialization only once in RecordWriter Key: FLINK-9913 URL: https://issues.apache.org/jira/browse/FLINK-9913 Project: Flink Issue Type: Improv