Hi all,
We are trying to use more than one count operator for dataset, but it executes
first count and skips other operations. Also we call env.execute().
How can we solve this problem?
-Ebru
Thank you Fabian, we’ve implemented a custom CsvInputFormat.
> On 24 Nov 2017, at 15:35, Fabian Hueske wrote:
>
> Hi Ebru,
>
> this case is not supported by Flink's CsvInputFormat. The problem is that
> such a file could not be read in parallel because it is not
n we solve this problem?
-Ebru
Flavio and Fabian thanks for your quick answers, it was very helpful.
-Ebru
> On 22 Nov 2017, at 20:47, Flavio Pompermaier wrote:
>
> I usually refer to this:
>
> https://github.com/FelixNeutatz/parquet-flinktacular
> <https://github.com/FelixNeutatz/parquet-flinktacular
Hello all,
We are trying to write dataset as parquet format, we use
AvroParquetOutputFormat but it is not compatible with Flink’s FileOutputFormat.
Is there a way to write dataset as parquet?
-Ebru
EBRU ÇETİNKAYA EBRU
wrote:
On 2017-11-10 13:14, Piotr Nowojski wrote:
jobmanager1.log and taskmanager2.log are the same. Can you also
submit
files containing std output?
Piotrek
On 10 Nov 2017, at 09:35, ÇETİNKAYA EBRU ÇETİNKAYA EBRU
wrote:
On 2017-11-10 11:04, Piotr Nowojski wrote:
Hi
On 2017-11-08 18:30, Piotr Nowojski wrote:
Btw, Ebru:
I don’t agree that the main suspect is NetworkBufferPool. On your
screenshots it’s memory consumption was reasonable and stable: 596MB
-> 602MB -> 597MB.
PoolThreadCache memory usage ~120MB is also reasonable.
Do you experien
On 2017-11-08 15:20, Piotr Nowojski wrote:
Hi Ebru and Javier,
Yes, if you could share this example job it would be helpful.
Ebru: could you explain in a little more details how does your Job(s)
look like? Could you post some code? If you are just using maps and
filters there shouldn’t be any
Hi Javier,
It would be helpful if you share your test job with us.
Which configurations did you try?
-Ebru
> On 8 Nov 2017, at 14:43, Javier Lopez wrote:
>
> Hi,
>
> We have been facing a similar problem. We have tried some different
> configurations, as proposed in oth
Hi Ufuk,
We don’t explicitly define any state descriptor. We only use map and filters
operator. We thought that gc handle clearing the flink’s internal states.
So how can we manage the memory if it is always increasing?
- Ebru
> On 7 Nov 2017, at 16:23, Ufuk Celebi wrote:
>
> Hey
Hi,
We are using Flink 1.3.1 in production, we have one job manager and 3
task managers in standalone mode. Recently, we've noticed that we have
memory related problems. We use docker container to serve Flink cluster.
We have 300 slots and 20 jobs are running with parallelism of 10. Also
the
11 matches
Mail list logo
