Re: Flink 1.1.3 RollingSink - understanding output blocks/parallelism

2016-12-15 Thread Aljoscha Krettek
Hi Dominik, I think having a single output file is only possible if you set the parallelism of the sink to 1. AFAIK it is not possible to concurrently write to a single HDFS file from multiple clients. Cheers, Aljoscha On Wed, 14 Dec 2016 at 20:57 Dominik Safaric wrote: > Hi everyone, > > altho

Flink 1.1.3 RollingSink - understanding output blocks/parallelism

2016-12-14 Thread Dominik Safaric
Hi everyone, although this question might sound trivial, I’ve been curious about the following. Given a Flink topology with parallelism level set to 6 for example and outputting the data stream to HDFS using an instance RollingSink, how is the output file structured? By structured, I refer to t